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MEMORANDUM  FOR 

DIRECTOR,  PROGRAM  ANALYSIS  AND  EVALUATION  (DPAE),  OFFICE  OF  THE 
CHIEF  OF  STAFF,  U.S.  ARMY,  WASHINGTON,  DC  20310-0200 
DEPUTY  CHIEF  OF  STAFF  FOR  OPERATIONS  AND  PLANS  (DCSOPS), 
WASHINGTON,  DC  20310-01*10 

SUBJECT:  Army  Program  Value  Added  Analysis  94-99  (VAA  94-99) 
Study 

1.  Reference  memorandum.  Office  of  the  Chief  of  Staff,  U.S. 
Army,  subject:  Army  Program  Value  Added  Analysis  94-99  (VAA 
94-99)  Study. 

2.  Referenced  memorandum  requested  that  the  U.S.  Army  Concepts 
Analysis  Agency  (CAA)  implement  the  methodology  developed  in 
the  Army  Program  Value  Added  Analysis'  -  Phase  I  Study.  This 
methodology  uses  a  hierarchical  assessment  framework  for 
determining  return  on  investment. 

3.  This  report  documents  the  results  of  our  analysis.  The 
principal  findings  of  the  study  are:  (1)  VAA  Study  framework 
as  developed  in  the  VAA  Phase  I  Study  was  shown  to  be  useful  in 
evaluating  Program  Objective  Memorandum  (POM)  issues;  (2)  The 
use  of  an  experimental  design  was  found  to  be  an  effective 
means  to  determine  system  contribution  to  combat  results;  (3) 
The  life  Cycle  Cost  Model  is  a  useful  tool  for  computing 
detailed  cost  estimates  for  candidate  procurement  programs;  (4) 
The  development  of  a  mixed  integer  programing  model  that  con¬ 
siders  cost  quantity  relationships  and  handles  fixed  production 
costs  and  research,  development,  test,  and  evaluation  (RDTE) 
explicitly  was  shown  to  be  an  extremely  effective  method  of 
cost-benefit  analysis;  and  (5)  Additional  research  is  required 
to  find  an  improved  method  of  effectiveness  integration. 

4.  This  Agency  expresses  appreciation  to  all  commands  and 
agencies  which  have  contributed  to  this  study.  Quencions 
and/or  inquiries  should  be  directed  to  the  Assistant  Director, 
Force  Systems  Directorate,  U.S.  Army  Concepts  Analysis  Agency, 
8120  Woodmont  Avenue,  Bethesda,  MD  20814-2797,  DSN  295-1546. 

r;  ^ 

E.  B.  VANDIVER  III 
Director 
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SUMMARY 

CAA-SR-92-10 

THE  REASON  FOR  PERFORMING  THE  STUDY  was  to  provide  the  Director  for  Program 
Analysis  and  Evaluation,  and  the  Deputy  Chief  of  Staff  for  Operations  and  Plans 
(DCSOPS)  an  analytical  methodology  and  capability  to  support  the  development  of  a 
balanced  and  effective  Army  Program. 

THE  STUDY  SPONSORS  are  the  Director  for  Program  Analysis  and  Evaluation 
(DPAE),  Office  of  the  Chief  of  Staff,  Army,  and  the  Technical  Advisor,  Office  of  the 
Deputy  Chief  of  Staff  for  Operations  and  Plans  (ODCSOPS),  Headquarters, 
Department  of  the  Army  (HQD  A). 

THE  STUDY  OBJECTIVES  were  to; 

(1)  Produce  VAA  coefficients  and  feasible  acquisition  alternatives  for  major  item 
systems  proposed  by  Headquarters,  Department  of  the  Army  (HQD  A)  Long-Range 
Research,  Development,  and  Acquisition  Plan  (LRRDAP)  and  constrained  by 
modernization  total  obligational  authority  (TOA).  The  process  must  measure  and 
analyze  the  capability  of  US  Army  forces  to  conduct  conventional  operations  in 
scenarios  consistent  with  the  Illustrative  Planning  Scenarios  of  the  Defense  Planning 
Guidance,  Fiscal  Year(FY)  1994-1999. 

(2)  Identify  and  develop  a  Value  Added  Analysis  Capability  (VAAC)  to  include  all 
appropriate  hardware,  software,  and  interfaces.  The  VAAC  must  tap  major 
authoritative  Army  data  bases  such  as  the  Total  Army  Equipment  Distribution 
Program  (TAEDP),  Force  Accounting  System  (FAS),  and  the  Army  Force  Cost 
System  (TAFCS), 

(3)  Identify  or  develop  models  and  techniques  that  support  the  VAA  methodology. 
The  VAAC  and  related  models  must  be  capable  of  operating  in  a  "quick  turnaround" 
environment,  defined  as  1  week  or  less. 

(4)  Conduct  a  demonstration  of  the  refined  methodology  and  VAAC  prior  to  the 
building  of  the  1994-1999  Program  Objective  Memorandum  (POM). 

(5)  Continue  the  refinement  and  implementation  of  the  VAA  methodology  for 
estimating  the  value  of  either  competing  major  item  systems  or  management 
decision  packages  (MDEPs)  to  the  Total  Army  Program. 

THE  SCOPE  OF  THE  STUDY  included  the  research,  development,  and  acquisition 
(RDA)  appropriation  for  selected  major  item  systems  in  FY  1994  and  FY  2008. 

THE  MAIN  ASSUMPTION  of  this  study  is  that  HQDA  needs  a  relatively  quick  method 
for  conducting  program  tradeoffs  which  has  sound  analytical  underpinnings. 


THE  BASIC  APPROACH  of  this  study  was  to; 

(1)  Enhance  and  expand  the  analytic  approach  for  program  issue  tradeoffs 
developed  in  Phase  I  of  the  VAA  study  effort. 

(2)  Develop  a  VAAC  for  implementing  the  methodology  to  include  software 
modules  where  appropriate. 

(3)  Demonstrate  the  VAAC  using  issues  from  the  94-99  POM  issue  cycle. 

THE  PRINCIPAL  FINDINGS  of  the  study  were; 

(1)  The  Value  Added  Analysis  study  framework,  as  developed  in  the  VAA  Phase  I 
Study,  was  shown  to  be  useful  in  evaluating  POM  issues. 

(2)  The  use  of  an  experimental  design  and  a  response  surface  methodology  was 
found  to  be  an  effective  means  to  determine  system  contribution  to  combat  results. 

(3)  The  Life  Cycle  Cost  Model  (LCCM)  is  a  useful  tool  for  providing  action  officers 
with  detailed  cost  estimates  for  candidate  POM  programs. 

(4)  The  development  of  a  mixed  integer  programming  formulation  that  allowed 
consideration  of  cost/quantity  relationships  and  handles  fixed  production  costs  and 
research,  development,  test,  and  evaluation  costs  explicitly  was  shown  to  be  an 
extremely  effective  method  of  cost-benefit  analysis. 

(5)  Additional  research  is  required  to  find  an  improved  method  of  effectiveness 
integration  to  replace  the  Technique  for  Order  Preference  by  Similarity  to  Ideal 
Solution  (TOPSIS)  Model. 

THE  STUDY  EFFORT  was  directed  by  LTC  Robert  R.  Koury,  Force  Systems 
Directorate,  US  Army  Concepts  Analysis  Agency  (CAA). 

COMMENTS  AND  QUESTIONS  may  be  sent  to  the  Director,  US  Army  Concepts 
Analysis  Agency,  ATTN:  CSCA-FSR,  8120  Woodmont  Avenue,  Bethesda,  Maryland 
20814-2797. 


Tear-out  copies  of  this  synopsis  are  at  back  cover. 
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CHAPTER  1 

EXECUTIVE  SUMMARY 


1-1.  PROBLEM.  The  leadership  at  Headquarters,  Department  of  the  Army 
(HQDA)  needs  analysis  to  support  the  development  of  a  balanced  and  effective  Army 
program  that  is  within  Department  of  Defense  IDOD)  resource  guidance. 

1-2.  STUDY  PURPOSE.  The  purpose  of  the  Army  Program  Value  Added  Analysis  - 
Phase  n  (VAA  94-99)  Study  was  to:  (1)  enhance  the  analytical  tradeoff  methodology 
developed  in  Army  Program  Value  Added  Analysis  90-97  Phase  I;  and  (2)  provide  the 
Director  for  Program  Analysis  and  Evaluation  (DPAE)  and  the  Deputy  Chief  of  Staff 
for  Operations  and  Plans  (DCSOPS)  with  an  analytical  capability  that  would  assist 
in  the  development  of  a  balanced  and  effective  Army  research,  development,  and 
acquisition  (RDA)  program  through  the  use  of  a  comprehensive  cost-benefit  analysis 
methodology. 

1-3.  BACKGROUND 

a.  Traditionally,  Army  program  development  is  accomplished  during  the 
Planning,  Programming,  Budgeting,  and  Execution  System  (PPBES)  Program 
Objective  Memorandum  (POM)  building  process  using  some  form  of  functional  or 
mission  area  panels.  The  processes  that  evolved  do  not  include  adequate  means  of 
integrating  these  functional  subprograms  into  a  balanced  total  Army  program. 
Furthermore,  the  traditional  POM  process  allowed  each  panel  to  use  its  own  methods 
for  prioritization.  This  lack  of  standardization  prevents  the  senior  leadership  from 
making  the  most  effective  tradeoffs  across  functional  areas. 

b.  Virtually  all  analysis  currently  performed  in  program  evaluation  focuses  on 
defining  individual  management  decision  package  (MDEP)  issues.  When  MDEP 
analysis  is  conducted,  the  senior  Army  leadership  lacks  the  visibility  or  analysis 
necessary  to  help  identify  the  marginal  value  of  resources  within,  or  across,  MDEPs. 

c.  Analysis  of  the  total  Army  program  requires  an  understanding  of  how 
individual  MDEPs  contribute  to  the  Army  mission  and  strategy  so  as  to  determine 
which  MDEPs  and  resource  levels  have  the  greatest  return  on  investment.  One 
approach  to  this  problem  is  to  estimate  the  "value  added"  by  individual  MDEPs,  or 
groups  of  MDEPs-expressed  as  program  enhancement  packages  or  alternative 
solutions- to  the  total  Army  program  as  measured  by  their  contribution  to  Army 
objectives. 

d.  The  Value  Added  Analysis  concept  uses  a  family  of  models  to  measure  an 
issue's  explicit  (objective)  contribution  to  the  program  as  an  incremental  or 
decremental  change  from  the  current  program  base.  A  survey  technique  is  used  to 
develop  an  issue's  implicit  (subjective)  contribution  to  the  program  through  the 
development  of  an  individual  alternative's  scores.  Saaty's  analytical  hierarchy 
technique  is  used  to  provide  a  structure  for  developing  weights  for  both  the  explicit 
and  implicit  measures  of  value.  Following  a  detailed  analysis  of  life  cycle  costs,  a 
cost-benefit  analysis  is  performed  usi  ng  an  optimization  model  to  determine  the  most 
cost  effective  acquisition  strategy  for  the  candidate  systems. 
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1-4.  STUDY  OBJECTIVES  were  to: 

a.  Produce  measures  of  the  worth  of  the  candidate  systems  (VAA  coefficients)  and 
feasible  acquisition  alternatives  for  major  item  systems  proposed  by  the 
Headquarters,  Department  of  the  Army  Long-Range  Research,  Development,  and 
Acquisition  Plan  (HQDA  LRRDAP)  and  constrained  by  modernization  total 
obligational  authority  (TO  A).  The  process  must  measure  and  analyze  the  capability 
of  US  Army  forces  to  conduct  conventional  operations  in  scenarios  consistent  with  the 
Illustrative  Planning  Scenarios  of  the  Defense  Planning  Guidance,  fiscal  years  (FYs) 
94-99. 

b.  Identify  and  develop  a  Value  Added  Analysis  Capability  (VAAC)  to  include  all 
appropriate  hardware,  software,  and  interfaces.  The  VAAC  must  tap  major 
authoritative  Army  data  bases  such  as  the  Total  Army  Equipment  Distribution 
Program  (TAEDP),  Force  Accounting  System  (FAS),  and  The  Army  Force  Cost 
System  (TAFCS). 

c.  Identify  or  develop  models  and  techniques  that  support  the  VAA  Methodology. 
The  VAAC  and  related  models  must  be  capable  of  operating  in  a  "quick  turnaround" 
environment,  defined  as  1  week  or  less. 

d.  Conduct  a  demonstration  of  the  refined  methodology  and  VAAC  prior  to  the 
building  of  the  1994-1999  POM. 

e.  Continue  the  refinement  and  implementation  of  the  VAA  methodology  for 
estimating  the  value  added  of  either  competing  major  item  systems  or  MDEPs  to  the 
total  Army  program. 

1-5.  STUDY  SCOPE 

a.  Baseline  program  is  President's  Budget,  FY  92,  and  US  Program  Force  (FY 
95). 

b.  The  analysis  examines  the  RDA  appropriations  and  other  related 
appropriations  as  needed  to  investigate  the  full  programing  of  selected  major  item 
systems. 

c.  About  50  major  item  systems  based  on  the  following  selection  criteria:  systems 
without  a  mission  needs  statement  (MNS)  or  other  requirements  document  (such  as  a 
required  operational  capability  (ROC))  as  of  1  October  1991  will  not  be  included, 
systems  with  large  dollar  amounts  in  current  and  proposed  programs  will  be 
included,  and  finally,  inclusion  of  a  system  is  subject  to  the  ability  to  execute  in  terms 
of  data  and  modeling  availability. 

d.  Scenario  Conditions.  Xorth  Atlantic  Treaty  Organization  (NATO)  Central 
Europe,  Post-CFE  (Conventional  Fc  rces  Europe),  Defense;  Southwest  Asia  (SWA) 
Defense  with  Counterattack;  Northeast  Asia  (NEA)  CounterofTensive;  consistent 
with  US  Army  Training  and  Doctrine  Command  (TRADOC)  scenario(s). 

e.  ConflictType.  Conventional 
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1-6.  STUDY  LIMITATIONS 

a.  Since  this  study  is  a  proof  of  concept  demonstration,  the  analysis  included  only 
the  RDA  appropriations.  Because  the  modeling  and  effectiveness  data  for  the 
noncombat  issues  are  less  developed  than  the  combat  areas,  the  study  focused 
principally  on  combat  tradeoff  issues. 

b.  The  original  list  of  equipment  issues  for  this  study  included  23  systems. 
Limited  data  available  for  combat  modeling  did  not  permit  the  inclusion  of  more  than 
12  systems  in  the  analysis. 

c.  RDA  appropriations  provide  only  22  percent  of  the  Army  program  dollars. 

1-7.  TIMEFRAME.  US  Army  conventional  operations  capabilities  will  be  assessed 
in  terms  of  force  packages,  major  item  systems  programmed  for  acquisition,  and 
anticipated  threat  as  of  the  end  of  FY  96,  FY  01,  and  FY  08. 

1-8.  KEY  ASSUMPTIONS 

a.  The  President's  FY  92  Budget  position  will  be  used  for  both  the  Program 
Optimization  and  Budget  Evaluation  (PROBE)  and  Research  Development  and 
Acquisition  Information  System  Agency  (RDAISA)  data  bases. 

b.  The  data  survey  results  are  assumed  to  reflect  decisionmaker  positions  and 
provide  a  means  of  modeling  the  effect  of  the  senior  Army  leadership  on  the  Army 
decision  making  process. 

c.  There  is  a  2-year  lag  from  expenditures  of  procurement  dollars  to  item  fielding. 
1-9.  STUDY  METHODOLOGY 

a.  Introduction.  The  VAA  methodology  was  developed  by  the  US  Army 
Concepts  Analysis  Agency  (CAA)  to  provide  optimized  acquisition  strategies  across 
system  types,  as  well  as  other  analysis  to  support  decisionmaking  necessary  to  build 
the  Army  budget.  The  VAA  methodology  is  modular,  and  each  module  performs  a 
specific  function.  Different  tools  can  be  used  to  perform  the  function  of  each  module 
depending  upon  the  analytical  requirements  established  by  the  issue  to  be  examined. 
Figure  1-1  shows  the  various  modules  and  their  interrelation.  A  brief  description  of 
the  modules  follows. 
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Figure  1-1.  Value  Added  Analysis  Modules 

b.  Issue  Definition.  The  purpose  of  the  Issue  Definition  Module  is  to  refine  the 
problem  and  its  associated  elements  to  be  studied  so  that  the  data  collection  and 
analysis  efforts  can  be  focused  on  the  questions  and  issues  of  interest  to 
decisionmakers.  Issue  definition  is  a  process  that  continues  for  the  duration  of  the 
Value  Added  Analysis.  It  establishes  the  general  context  of  the  study  in  terms  of  the 
systems  and  programs  to  be  analyzed,  as  well  as  timeframes  and  scenarios  of  interest. 
The  module  also  encompasses  the  process  of  clarifying  the  specific  questions  asked  by 
the  decisionmakers. 

c.  Explicit  and  Implicit  Effectiveness.  Systems  efTectiveness  is  measured  in 
two  ways.  In  the  Explicit  Effectiveness  Module,  the  systems  of  interest  are  portrayed 
in  a  combat  simulation,  and  their  contribution  to  force  level  performance  is 
measured.  Not  all  pertinent  criteria  that  bear  on  the  procurement  decision  are 
measurable  in  this  manner.  The  purpose  of  the  Implicit  EfTectiveness  Module  is  to 
quantify  these  hard-to-measure  factors.  These  factors  might  include  political  risk, 
impact  on  sustainability,  and  programmatics,  as  well  as  other  criteria  that  cannot  be 
directly  measured  at  present.  Evaluations  are  made  by  individuals  who  are  experts 
in  these  criteria.  The  criteria  are  assigned  weights  of  relative  importance  based  on  a 
survey  of  senior  Army  decisionmakers,  bubject  matter  experts  then  evaluate  (score) 
how  well  a  system  fares  in  light  of  these  criteria. 

d.  Effectiveness  Integration.  When  the  programs  of  interest  are  finally 
evaluated,  a  vector  of  effectiveness  scores  for  the  various  criteria  is  obtained  for  each 
system.  The  purpose  of  the  Effectiveness  Integration  Module  is  to  reduce  this  vector 
of  information  to  a  single  measure.  The  Technique  for  Order  Preference  by 
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Similarity  to  Ideal  Solution  (TOPSIS)  is  currently  used  for  this  purpose.  This  method 
is  described  in  detail  by  Hwang  and  Yoon.  The  single  measure  obtained  for  each 
system  can  then  be  used  to  compare  the  systems  in  question  with  respect  to  their 
effectiveness.  These  measures  are  used  to  form  the  objective  function  coefficients  for 
the  VAA  optimization. 

e.  Cost.  Parallel  to  the  determination  of  the  effectiveness  of  the  system  in 
question  is  the  determination  of  system  costs.  For  the  purpose  of  developing  an 
acquisition  strategy,  the  portion  of  the  life  cycle  cost  of  interest  is  the  funds  that 
would  be  available  for  research,  development,  and  acquisition  (RD  A).  Data  are  also 
collected  describing  the  various  components  of  the  RDA  costs:  fixed  costs,  variable 
costs  with  learning  curve  effects,  and  variable  costs  without  learning.  These  are  the 
costs  that  are  used  to  build  the  budgetary  constraints  for  the  VAA  Optimization 
Module.  After  the  acquisition  strategies  are  developed,  accounting  and  analysis  of 
total  life  cycle  costs  are  then  conducted. 

f.  Optimization.  Previous  mixed  integer  programming  problems  have  been  used 
at  CAA  to  produce  acquisition  strategies  for  various  systems  such  as  helicopters  or 
trucks.  The  need  arose  to  provide  optimized  acquisition  strategies  across  system 
types,  and  to  use  of  nonlinear  cost  learning  curves,  which  more  accurately  represent 
system  costs  as  a  function  of  quantity  produced.  The  Value  Added  Linear 
Optimization  of  Resources  (VALOR)  Model,  a  mixed  integer  linear  programming 
formulation  that  uses  a  piecewise  linear  approximation  of  the  learning  curve  costs  for 
a  more  accurate  portrayal  of  budgetary  constraints,  was  developed  to  meet  this  need. 
The  objective  of  the  VAA  Optimization  Module  is  to  maximize  the  effectiveness  of  the 
force  subject  to  constraints  on  budget,  force  structure,  and  production  capability. 

1-10.  ESSENTIAL  ELEMENTS  OF  ANALYSIS  (EEA).  The  essential  elements  of 
analysis  and  basic  findings  for  each  are  as  follows. 

a.  EEA  1.  What  is  the  "value  added"  of  a  select  number  of  major  item 
systems  to  be  considered  in  the  1994-1999  Program  Objective  Memorandum 
(POM)? 

(1)  Initially  a  list  of  119  systems  was  reviewed  for  the  VAA  Phase  11  effort.  This 
list  was  revised  using  a  set  of  criteria  which  included  the  ability  to  model  the  system, 
the  existence  of  a  system  requirements  document,  and  the  determination  that  the 
system  represented  a  major  acquisition  in  terms  of  the  total  dollars  associated  with 
the  system's  program.  The  first  cut  of  the  original  119,  based  on  the  previously 
mentioned  criteria,  resulted  in  a  list  of  50  major  item  systems.  Each  of  the  systems 
on  this  "first  cut"  list  were  modeled  in  the  Corps  Battle  Analyzer  (CORE  AN).  As 
decisions  were  made  by  the  Army  leadership,  several  systems  were  dropped  from 
consideration.  The  final  list  consisted  of  41  systems. 

(2)  The  "value  added"  of  each  of  the  systems  by  year  (FY  96,  FY  01,  and  FY  08) 
given  as  a  1  to  N  list  developed  from  the  effectiveness  integration  module  was 
produced.  This  list  is  provided  in  Chapter  9. 
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b.  EEA  2.  What  kind  of  characteristics  should  the  VAAC  have  in  order  to 
implement  the  VAA  methodology? 

(1)  As  defined  in  the  VAA  Phase  I  Study,  the  VAAC  was  to  be  a  set  of  models 
and  tools  residing  on  the  METAPHOR  computer  architecture.  The  advantages  of  this 
arrangement  as  originally  developed  was  it  would  allow  the  action  officer  in  the 
Pentagon  to  use  the  data  base  engine  of  METAPHOR  to  define  and  shape  the  issues 
to  be  studied.  The  action  officer  would  then  use  the  ability  of  METAPHOR  to  move  in 
and  out  of  the  personal  computer  (PC)  environment  in  order  to  run  the  VAA  models 
and  tools  for  conducting  a  value  added  analysis.  CAA,  the  Program  Analysis  and 
Evaluation  Directorate  (PAED),  and  Office  of  the  Deputy  Chief  of  StafT  for 
Operations  and  Plans  (ODCSOPS)  Information  Management  Office  (IMO)  were  able 
to  obtain  connectivity  and  many  of  the  tools  and  unique  VAA  data  bases  were  placed 
in  the  METAPHOR  environment.  However,  the  ability  to  move  in  and  out  of  the 
METAPHOR  environment  into  the  PC  environment  was  found  to  be  somewhat 
limited.  This  limitation  coupled  with  time  and  resource  constraints  associated  with 
having  action  officer  attempt  to  do  these  types  '’f  analyses  prohibited  the 
implementation  of  the  proposed  VAAC  design  during  the  study  period. 

(2)  It  was  determined  that  the  action  officers  in  the  Pentagon  were  unable  to 
find  the  time  to  learn  either  the  METAPHOR  computer  software  or  the  VAA  tools. 
This  constraint  indicated  that  the  VAAC  would  have  to  be  designed  to  allow  the 
action  officers  access  to  the  input  data,  issue  definition  narrative  data,  and  then  to 
the  results  of  the  value  added  analysis.  However,  the  value  added  analysis  would 
need  to  be  conducted  by  an  analytical  organization  outside  of  the  HQDA  Army  Staff 
(ARSTAF)  or  by  a  special  study  group.  The  need  to  have  a  separate  study  group  and 
visibility/access  for  the  action  officers  creates  the  requirement  for  the  VAAC  to  be 
able  to  move  easily  from  a  data  base  environment  into  a  decentralized  computational 
environment  and  back  again.  The  ability  to  pull  data  from  the  Army  corporate  data 
bases,  format  the  data,  and  then  move  that  data  into  the  VAA  tools  and  models  must 
be  well  developed.  The  connectivity  of  data  and  tools  must  be  enhanced  over  the 
manner  in  which  this  type  of  function  was  handled  in  VAA  Phase  H,  which  was  the 
hand-carrying  of  data  on  disks  from  one  machine  to  another.  The  study  group 
conducting  the  analysis  should  be  able  to  move  from  the  IBM  RISC  6000  computer  to 
the  Macintosh  computer  and  back  to  the  data  bases  in  manner  which  is  transparent 
to  the  analyst.  Likewise,  the  action  officer  must  be  able  to  query  the  results  of  the 
study  group  work,  the  assumptions  for  the  work,  and  the  input  data  for  the  particular 
work. 

c.  EEA  3.  What  kinds  of  tools,  techniques,  and  models  are  needed  to 
support  the  VAA  methodology? 

(1)  The  VAA  Phase  I  effort  investigated  the  idea  of  using  spreadsheet  costing 
tools,  a  simplified  optimization,  and  the  use  of  CORB  AN  for  the  combat  modeling. 
Much  of  the  work  that  was  completed  during  Phase  I  was  hand  generated  with  little 
experience  as  to  how  data  or  processes  might  be  connected.  The  major  thrust  of  the 
VAA  Phase  II  effort  was  to  develop  a  more  complete  combat  modeling  capability, 
expand  the  optimization  tool,  implement  the  changes  to  TOPSIS  recommended  by  the 
Operations  Research  Systems  Analysis  (ORSA)  Center  at  the  US  Military  Academy 
(USMA),  and  take  the  cost  analysis  concepts  and  grow  them  into  a  full  set  of  tools. 

The  survey  technique  developed  during  VAA  Phase  I  and  the  use  of  the  analytical 
hierarchy  process  remained  relatively  unchanged.  The  only  changes  associated  with 
these  last  two  techniques  was  the  development  of  a  different  hierarchy.  This  was  not 
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a  matter  of  change  in  technique  as  much  as  it  was  defining  the  context  for  the  VAA 
Phase  n  effort.  Essentially,  the  survey  approach  was  the  same  in  both  phases. 

(2)  The  first  model  improvement  needed  to  support  the  VAA  methodology 
included  a  combat  model  which  was  both  easier  to  set  up  and  faster  in  its  run  time 
than  the  stochastic  CORBAN  used  in  the  Phase  I  study.  The  team  looked  at  other 
models  such  as  Analysis  of  Force  Potential  (AFP),  Eagle,  and  Vector-In-Commander 
(VIC)  as  possible  solutions.  It  was  decided  that  AFP  was  at  too  aggregated  a  level, 
that  Eagle  was  not  yet  developed  enough  to  use,  and  that  VIC  was  at  too  high  a  level 
of  resolution.  All  of  these  models  also  had  the  same  limitations  as  CORBAN  in  that 
they  all  have  extensive  setup  requirements.  This  review  of  the  combat  models  had 
the  net  effect  of  validating  the  need  for  CORBAN.  However,  the  team  determined 
that  a  full  factorial  design  using  stochastic  CORBAN  was  not  possible.  Therefore, 
the  team  worked  with  TRADOC  to  obtain  a  deterministic  version  of  CORBAN  in  an 
effort  to  speed  up  the  run  time.  This  change  proved  to  be  extremely  important 
because  in  allowed  the  team  to  perform  excursions  relatively  rapidly.  However,  in 
order  to  develop  values  for  all  41  systems,  a  very  large  experimental  design  was 
required.  Even  a  deterministic  CORBAN  was  not  fast  enough  to  accomplish  all  the 
runs  required.  The  requirement  to  perform  many  excursions  for  a  large  range  of 
weapon  system  mixes  led  to  the  use  of  a  response  surface  methodology  (RSM).  The 
VAA  Phase  H  experience  in  using  RSM  was  very  favorable.  The  RSM  technique 
allowed  the  team  to  accomplish  the  development  of  values  for  all  of  the  systems.  RSM 
also  had  a  side  benefit  in  allowing  the  team  to  predict  the  explicit  measure  of 
effectiveness  (MOE)  values  one  would  obtain  by  playing  the  weapon  mix  coming  out 
of  the  optimization. 

(3)  A  more  fully  developed  optimization  formulation  was  another 
technique/model  needed  to  fully  implement  the  VAA  methodology.  In  VAA  Phase  I, 
a  relatively  simple  formulation  was  used  for  the  linear  pro^am  in  order  to 
demonstrate  the  feasibility  of  the  methodology.  It  was  evident  that  any  follow-on 
effort  had  a  need  for  dynamic  cost-quantity  modeling  to  accommodate  the  concept  of 
system  learning  curves  in  order  to  provide  a  more  accurate  cost  within  the 
optimization.  Phase  I  also  identified  the  requirement  to  link  certain  systems 
together  if  they  are  dependent  upon  one  another,  for  example  Advanced  Field 
Artillery  System  (AFAS)  and  future  armored  resupply  vehicle  -  artillery  (FARV-A), 
where  FARV-A  is  not  procured  unless  AFAS  is  being  procured.  Finally,  the  capacity 
to  continue  to  fully  incorporate  both  force  structure  and  production  data  was 
maintained,  along  with  institution  of  better  quality  control  on  the  data. 

(4)  VAA  Phase  I  validated  the  concept  of  using  a  linear  combination  technique 
for  integrating  the  effectiveness  values.  The  TOPSIS  technique  used  in  Phase  I  was 
reviewed  by  the  ORSA  Center  at  USMA.  Their  suggestions  for  improvement  were 
implemented  in  the  Phase  H  Study. 

(5)  The  cost  analysis  tools  developed  as  part  of  this  VAA  Phase  II  effort  by  the 
sponsors  have  proven  to  to  be  very  useful.  The  Life  Cycle  Cost  Model  (LCCM)  has 
been  expanded  and  refined  to  include  the  ability  to  conduct  "what  ifs"  on  quantities 
and  average  unit  costs.  The  tool  has  been  transferred  to  HQDA  PAED  and  is 
currently  in  use  by  PAED  analysts.  Further  refinement  to  include  the  development 
of  generic  LCCMs  by  mission  area  would  be  useful  in  conducting  analysis  on 
developmental  systems  which  currently  do  not  have  a  baseline  cost  estimate  (BCE). 
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d.  EE  A  4.  What  explicit  effectiveness  models  are  most  appropriate  for  a 
Value  Added  Analysis  Study  (V^alue  Added  Analysis  1994-1999)? 

( 1)  In  VAA  Phase  I,  the  issue  of  using  a  highly  aggregated  combat  model  was 
addressed.  The  conclusion  was  that  an  aggregated  model  could  provide  useful 
insights  for  answering  macro  level  program  tradeoff  questions.  However,  it  was  also 
noted  that  decisionmakers  require  sufTicient  detail  to  understand  the  reasons  for 
combat  outcomes  to  assist  them  in  specific  program  decisions.  This  point  was 
reinforced  in  work  completed  as  part  of  VAA  Phase  H.  Almost  universally,  decision¬ 
makers  wanted  to  know  the  reasons  behind  the  combat  results.  The  use  of  RSM 
compounded  the  problem  by  not  considering  the  effects  of  interaction  and  only 
looking  at  the  first  order  effects.  This  tradeoff  be  tween  detail  and  the  ability  to  look 
at  a  large  number  of  systems  will  continue  to  be  an  issue. 

(2)  This  particular  EE  A  was  aimed  at  a  larger  issue  than  just  the  research, 
development’,  and  acquisition  (RDA)  appropriation  modeling  and  the  combat 
modeling  associated  with  study  weapon  system  tradeoffs.  At  the  outset,  VAA  Phase 
n  was  starting  to  look  at  other  appropriations  such  as  Operation  and  Maintenance 
Army  (OMA),  Military  Construction  Army  (MCA),  and  perhaps  Military  Personnel 
Army  (MPA).  Although  these  appropriations  were  considered,  especially  in  the 
LCCM  costing,  true  tradeoffs  were  not  conducted.  The  team  did  explore  some 
preliminary  tradeoffs  in  the  area  of  OMA  to  RDA  and  MPA  to  RDA.  This  prelimi¬ 
nary  work  suggests  that  additional  explicit  effectiveness  models  are  required,  such  as 
operational  tempo  (OPTEMPO)  models,  training  models,  and  the  inclusion  of 
campaign  models.  This  is  an  area  which  requires  additional  research  as  part  of  a 
follow-on  study. 

e.  EEA5.  What  refinements  are  needed  to  improve  the  VAA  methodology? 
Answers  to  the  previous  EEA  suggest  some  areas  for  refinements  already.  The  work 
completed  in  the  VAA  Phase  II  effort  has  shown  that  the  methodology  works  and  can 
provide  useful  and  timely  support  to  decisionmakers.  Additional  work  is  needed  to 
expand  the  use  of  the  methodology  into  a  broader  range  of  issues.  These  include  a 
variety  of  combat  systems  (e.g.,  logistics  and  command  and  control  (C2))  and  other 
appropriations  (e.g.,  OMA  and  MCA). 

1-11.  OTHER  KFl  Y  FINUI.NGS.  The  observations  and  findings  contained  in  this 
paragraph  are  not  keyed  to  the  original  questions  proposed  in  the  study  directive,  but 
are  insights  obtained  as  a  result  of  the  work  completed  in  this  study. 

a.  Input  data  intensity,  accuracy,  and  timeliness  continue  to  be  of  crucial  impor¬ 
tance  in  making  the  value  added  process  operational. 

b.  The  need  to  have  the  survey  process  standardized  was  confirmed  in  this  study. 

c.  Issue  definition  is  clearly  an  iterative  process  and  should  not  be  thought  of  as 
having  a  beginning  or  ending,  per  se. 

d.  Future  experimental  designs  should  try  to  incorporate  the  need  to  investigate 
second  order  effects  (i.e.,  synergism  and  weapon  system  interactions). 
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e.  Production  data  as  depicted  in  the  HQDA  procurement  (P-21)  forms,  which  are 
used  to  provide  procurement  information  to  Congress,  are  not  sufficient  or  timely 
enough  to  conduct  Value  Added  Analysis  on  an  institutionalized  basis.  The  produc¬ 
tion  data  required  for  Value  Added  Analysis  (as  well  as  for  other  acquisition  analyses 
conducted  for  HQDA)  should  be  made  readily  available  and  current  in  support  of 
budgetary  decisionmaking. 

f.  Standardized  cost  codes  and  views  as  implemented  in  the  VAA  Phase  II  V alue 
Added  Analysis  Study  should  be  used  throughout  the  Army.  The  Life  Cycle  Cost 
Model  should  be  made  available  to  Program  Executive  Offices  (PEOs)/  Program 
Managers  (PMs)  and  help  form  the  basis  for  standardization. 

g.  Major  item  system  data  as  defined  in  the  Army  Resource  Integration  and 
Management  (ARIM)  Study  should  be  institutionalized  and  used  throughout  the 
procurement  process  to  assist  in  standardization  of  costs  and  system  definition. 
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CHAPTER  2 
INTRODUCTION 


2-1.  OVERVIEW 

a.  The  primary  purpose  of  this  chapter  is  to  provide  a  discussion  of  the 
background  which  influenced  the  conduct  of  the  Phase  11 VAA  Study.  Additionally, 
this  chapter  provides  a  discussion  of  the  limitations  and  assumptions  associated  with 
the  Phase  n  work  and  a  brief  overview  of  the  methodological  refinements  developed 
as  part  of  Phase  n. 

b.  The  purpose  of  the  Phase  n  Study  was  to  implement  the  VAA  methodology  as 
created  and  defined  in  the  Phase  I  work.  The  study  focused  on  refining  the  VAA 
methodology  and  actually  assisting  the  ARSTAF  in  allocating  resources  for 
modernization  programs.  The  latter  was  accomplished  by  using  the  VAA 
methodology  as  part  of  the  LRRDAP  review  process.  The  VAA  Phase  II  study 
directive  identified  five  objectives  to  be  accomplished  as  part  of  the  study.  In  meeting 
these  objectives,  the  study  produced  VAA  system  coefficients  and  a  series  of  feasible 
acquisition  alternatives.  The  study  identified,  and,  to  a  limited  degree,  developed  a 
Value  Added  Analysis  Capability  which  included  all  appropriate  hardware,  software, 
and  interfaces.  Finally,  the  study  team  identified  and  developed  models  and 
techniques  that  supported  the  VAA  methodology.  Chapter  2  is  structured  as  follows; 

•  Problem  Background 

•  A  Review  of  the  VAA  Phase  I  Findings 

•  VAA  Phase  II  as  an  Outgrowth  of  VAA  Phase  I 

•  Methodology  Overview 

•  Methodological  Refinements 

•  Summary 

2-2.  PROBLEM  BACKGROUND 

a.  The  growing  federal  budget  deficit  and  the  breakout  of  peace  in  Europe  have 
put  great  pressure  on  all  of  the  armed  services  to  reevaluate  and  restructure  their 
budget  and  programs.  This  pressure  is  especially  great  on  the  Army  because  of  the 
changing  perception  of  the  threat  in  Western  Europe.  There  is  more  need  now  for 
cost-benefit  analysis  than  ever  before.  This  analysis  must  be  conducted  in  support  of 
developing  a  balanced  and  effective  Anny  program  within  Department  of  Defense 
resource  guidance.  Traditionally,  the  Army  has  used  functional  area  panels  to  build 
its  POM.  The  processes  that  have  evolved  do  not  include  adequate  means  of 
integrating  and  balancing  the  functional  programs  into  the  total  Army  program.  The 
Value  Added  Analysis  has  been  conceived  as  a  means  of  assisting  in  accomplishing 
this  integration  and  balancing. 

b.  The  environment  in  which  Value  Added  Analysis  must  be  conducted  is 
characterized  by  minimal  time  for  analysis,  ever-changing  assumptions,  often 
incomplete  data,  and  increasing  emphasis  on  verifiable  results.  Traditional  staff 
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processes  and  supporting  operations  resparch  studies  often  do  not  meet  this 
challenge.  This  failure  occurs  because  traditional  approaches  require  a  long  time  and 
highly  trained  personnel  to  produce  high  quality,  fully  integrated,  and  very  detailed 
analysis. 

c.  The  requirement  for  a  Value  Added  Analysis  methodology  was  conceived  by 
the  Director,  Program  Analysis  and  Evaluation,  as  a  means  of  conducting  program 
tradeoff  analyses.  In  early  1989,  DPAE,  asked  the  Technical  Advisor,  DCSOPS,  to 
jointly  sponsor  a  study  to  be  conducted  by  CAA  to  develop  this  methodology.  The 
subsequent  study  was  called  Army  Program  Value  Added  Analysis  90-97  (VAA  90- 
97)  and  was  completed  in  August  of  1991.  The  primary  purpose  of  the  VAA  Phase  I 
Study  was  to  develop  and  demonstrate  a  methodology  for  conducting  marginal  cost- 
benefit  analyses. 

2-3.  A  REVIEW  OF  THE  VAA  PHASE  I  FINDINGS 

a.  Background.  The  VAA  Phase  I  Study  was  designed  as  a  proof  of  concept  to 
include  the  building  of  a  prototype  decision  support  system.  Because  the  work  was 
developmental,  the  study  team  limited  the  scope  to  weapon  system  tradeoffs  and  did 
not  attempt  to  address  force  structure,  personnel,  infrastructure,  or  other  issues.  The 
methodology  was,  however,  designed  to  be  generic  and  used  across  all  the  functional 
areas  of  the  Army  program. 

b.  Primary  Observations  and  Findings.  The  primary  finding  of  the  VAA 
Phase  I  Study  was  the  definition  of  a  flexible  and  rational  methodology  to  support  the 
POM  process  and  Army  program  development.  However,  the  study  team  was  not 
able  to  completely  automate  the  decision  support  system.  The  study  team  found  that 
a  linear  combination  of  value  components  creating  a  single  measure  of  an  issue's 
marginal  value  was  effective  in  ranking  alternatives  and  conducting  tradeoff  analy¬ 
ses.  However,  the  techniques  available  for  doing  this  combination  have  several 
analytical  weaknesses  including  scale  compression,  inadequate  treatment  of  equiva¬ 
lence  classes,  and  sensitivity  to  probabilistic  inputs.  The  methodology  developed 
during  VAA  Phase  I  was  successful  in  capturing  the  subjective  elements  used  by 
decisionmakers  in  conducting  tradeoffs  between  alternatives  through  the  use  of 
judgment  weights.  The  VAA  Phase  I  Study  also  provided  some  important  insights 
into  the  use  of  aggregated  effectiveness  models,  new  costing  approaches,  and 
optimization  techniques  which  were  built  upon  in  this  Phase  II  Study.  The  following 
is  an  abbreviated  list  and  discussion  of  the  EEAs,  observations,  and  findings: 

(1)  Two  generalized  categories  of  measures  (explicit  and  implicit  effectiveness) 
were  discovered  to  be  important  in  judging  relative  value. 

(2)  Viewing  system  effectiveness  as  an  integrated  piece  of  force  effectiveness, 
instead  of  measuring  pure  weapon  system  performance,  yields  a  more  rational  view 
of  weapon  system  contribution  across  systems  and  mission  areas. 

(3)  Implicit  effectiveness  factors  (secondary  impact  analysis  modifiers  (SIAM, 
factors)  beyond  explicit  (pure  combat)  effectiveness  are  a  significant  component  of  the 
Headquarters,  Department  of  the  Army  (HQDA)  decision  process,  and  a  management 
survey  can  yield  important  inputs  into  this  process. 

(4)  No  one  single  measure  of  combat  effectiveness  is  adequate  to  measure  the 
contribution  of  weapon  systems.  Establishing  a  hierarchy  of  MOE  was  found  to  be  an 
effective  way  to  use  results  from  different  models. 
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(5)  The  systematic  specification  of  costs  and  benefits  within  a  consistent  frame¬ 
work  understood  by  both  analyst  and  decisionmaker  is  an  extremely  positive  feature 
of  the  methodology. 

c.  Other  Observafions  and  Findings.  The  observations  and  findings  contained 
in  this  para^aph  are  not  necessarily  keyed  to  the  original  questions  proposed  in  the 
study  directive  but  are  insights  obtained  as  a  result  of  the  work  completed  in  the 
VAA  Phase  I  Study. 

(1)  The  Value  Added  methodology  is  data  intensive.  Accuracy  and  timeliness  of 
input  data,  especially  production  data,  is  of  crucial  importance  in  making  the  value 
added  process  operational. 

(2)  The  initial  goal  of  value  added  was  to  be  quick  turnaround  for  all 
components.  What  was  found  was  that  an  extensive  front-end  process  (especially 
combat  modeling  and  costing)  was  required  which  consumed  large  amounts  of  time. 

(3)  The  Value  Added  methodology  will  "buy"  the  most  cost  effective  systems 
subject  to  constraints. 

(4)  The  study  developed  a  modular  "living"  methodology  with  extreme 
flexibility  that  can  be  used  with  a  variety  of  techniques  and  models. 

(5)  Issue  clarification  is  difficult,  and  it  appears  that  senior  decisionmakers 
have  difficulty  focusing  issues. 

(6)  The  technique  used  to  survey  the  Army  leadership  allows  decision  making 
attitudes  and  behaviors  to  be  accurately  modeled, 

2-4.  VAA  PHASE  II  AS  AN  OUTGROWTH  OF  VAA  PHASE  I 

a.  Upon  completion  of  the  VAA  Phase  I  Study,  an  evaluation  of  the  results  was 
conducted  to  determine  the  direction  of  VAA  Phase  11.  The  Value  Added  Analysis 
study  effort  was  originally  planned  as  having  two  parts.  The  study  team  and 
sponsors  intended  to  have  Phase  I  develop  the  methodology  and  Phase  II  implement 
that  methodology  within  HQDA  at  both  PAED  and  ODCSOPS.  The  view  was  that  a 
decision  support  system  would  be  implemented  on  the  METAPHOR  computer  which 
would  allow  action  officers  at  HQDA  to  frame  and  conduct  their  own  Value  Added 
Analyses.  As  indicated  in  the  previous  paragraph,  it  became  evident  to  the  study 
team  and  the  sponsors  that  a  quick  turnaround,  in-house  analysis  capability  would 
not  be  possible  as  originally  envisioned.  The  initial  view  of  implementation  had  the 
entire  data  collection  effort  and  combat  modeling  being  completed  at  HQDA.  The 
idea  that  an  analytical  agency  would  be  needed  to  conduct  the  analysis  was  not  part 
of  the  plan.  However,  as  the  study  team  gained  experience  and  a  better  under¬ 
standing  of  the  components  of  the  methodology  was  developed,  it  became  evident  that 
an  extensive  upfront  analysis  would  be  required.  This  finding  was  important  in 
shaping  the  VAA  Phase  II  Study  because  it  now  pointed  to  the  need  for  the 
continuing  involvement  of  an  analytical  agency  in  the  VAA  study  process. 

b.  The  VAA  Phase  1  finding  which  indicated  the  need  for  extensive  upfront  analy¬ 
sis  by  an  analytical  agency  helped  shape  a  new  view  of  the  division  of  labor  or  study 
effort  between  a  yet  unidentified  analytical  agency  and  the  action  officers  at  HQDA. 
This  finding  also  required  the  VAA  Phase  IT  implementation  effort  to  change 


2-3 


CAA-SR-92-10 


its  focus  slightly.  The  idea  of  implementing  a  complete  handofT  of  all  analysis  to  the 
action  ofiicer  changed  to  one  in  which  the  combat  modeling,  data  collection,  and 
optimization  formulation  would  be  completed  at  a  location  other  than  HQDA  and 
periodically  refreshed.  HQDA  would  be  involved  in  helping  shape  the  upfront 
analysis  and  would  use  the  data  bases  created  as  a  result  of  the  analysis  to  conduct 
their  inquiries  and  analysis.  Keeping  this  change  in  mind,  the  study  team  started  to 
develop  its  plan  for  implementation. 

c.  The  following  scope,  limitations,  and  assumptions  were  used  in  conducting  the 
VAA  Phase  11  Study: 

(1)  The  baseline  program  which  was  used  included  the  President's  Budget  FY 
91  and  US  Program  Force  as  of  FY  95. 

(2)  The  analysis  focused  primarily  on  the  research,  development,  and  acquisi¬ 
tion  appropriations  and  only  looked  at  other  appropriations  as  needed  in  order  to 
relate  back  to  the  RDA  accounts. 

(3)  The  analysis  examined  41  major  item  systems. 

(4)  The  analysis  considered  US  Army  conventional  operations  in  terms  of  force 
packages,  major  item  systems  programmed  for  acquisition,  and  anticipated  threat  as 
of  the  end  of  FY  96,  FY  01,  and  FY  08. 

(5)  Three  scenarios  were  considered  in  developing  the  combat  modeling  and 
analytical  hierarchy  to  include  a  NATO  Central  Europe  -  post-CFE  defense,  a  SWA 
defense  with  a  counterattack,  and  an  NEA  attack. 

(6)  The  VAAC  will  initially  be  installed  at  CAA,  but  would  be  eventually 
installed  at  PA&E  and  ODCSOPS. 
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d.  Although  the  focus  on  where  the  analysis  would  be  conducted  changed,  the 
basic  methodology  did  not.  It  was  determined  that  the  overall  methodology  as 
developed  in  Phase  I  was  still  viable,  and  that  the  methodology  only  required 
certain  refinements  in  order  to  implement  it  for  the  conduct  of  actual  analyses. 

2-5.  METHODOLOGY  OVERVIEW 

a.  The  first  objective  of  this  study,  as  outlined  in  the  study  directive,  was  to 
implement  the  VAA  methodology  as  created  and  defined  in  the  Phase  I  work.  This 
study  focused  on  refining  the  VAIA  methodology  and  actually  assisting  the  ARSTAF 
in  allocating  resources  for  modernization  programs.  The  VAA  methodology  has  been 
developed  to  provide  a  road  map  for  conducting  these  analyses.  The  methodology 
provides  a  generic  approach  for  cost-benefit  analysis.  Aa  important  feature  of  this 
methodology  is  that  it  is  flexible  and  may  be  used  in  both  a  standardized  or  ad  hoc 
approach.  The  VAA  standardized  approach  v^ould  use  the  complete  methodology  as 
described  in  this  chapter,  to  include  all  of  the  current  techniques  developed.  The  ad 
hoc  approach  uses  only  those  modules  and  techniques  pertinent  to  the  issue  being 
investigated.  This  flexibility  is  achieved  by  using  a  modular  methodological 
framework.  Figure  2-1  depicts  the  complete  "standardized”  VAA  methodology  with 
its  eight  modules. 


Figure  2-1.  Value  Added  Analysis  Methodology  Overview 
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b.  Although  the  VAA  methodology  is  designed  to  be  generic  and  to  be  used  with 
any  of  the  appropriations  (e.g.,  OMA,  MCA,  RDA,  etc.),  the  Phase  II  study  effort 
concentrated  primarily  on  the  RDA  accounts.  The  VAA  methodology  for  the  RDA 
appropriation  coasists  of  the  following  steps: 

(1)  Step  1  -  Issue  Definition.  Identify  and  define  issues.  Select  analytical 
tools  and  data  required  to  address  them. 

(2)  Step  2  -  Explicit  Effectiveness.  Using  a  combat  simulation  model, 
determine  the  relative  contribution  of  selected  systems  to  force-level  combat 
effectiveness. 

(3)  Step  3  -  Implicit  Effectiveness.  Based  upon  survey  input  from  subject 
matter  experts,  assign  scores  to  the  selected  systems,  indicating  their  rating  with 
respect  to  the  implicit  effectiveness  factors. 

(4)  Step  4  -  Effectiveness  Integration.  Based  upon  a  survey  of  senior  Army 
leaders,  derive  weights  for  factors  that  influence  the  decisionmaking  process.  The 
senior  Army  leader  survey  produces  weights  that  reflect  the  relative  importance  of 
different  combat  scenarios  (for  selected  years)  and  explicit  and  implicit  effectiveness 
factors  in  the  decisionmaking  process.  A  mathematical  prioritization  tool  combines 
the  results  of  the  combat  model  and  the  subject  matter  expert  and  senior  Army 
leadership  surveys  and  produces  a  list  of  systems  rank-ordered  according  to  their 
relative  effectiveness,  as  well  as  a  numerical  measure  of  that  effectiveness. 

(5)  Step  5  -  Costing.  Life  cycle  costs  which  include  materiel,  personnel,  and 
system-specific  facility  expenses  are  produced  for  the  selected  systems.  Several  cost 
looks  can  be  produced  depending  on  user  need. 

(6)  Step  6  -  Optimization.  System  effectiveness  and  costs  are  evaluated  in  a 
mathematical  optimization  model  that  develops  an  affordable,  feasible  program 
alternative-i.e.,  what  should  be  bought,  how  many,  and  when--while  accounting  for 
budget  and  production  constraints,  force  structure  requirements,  production 
capabilities,  and  learning  curve  effects. 

(7)  Step  7  -  Resource  Allocation.  System  distribution  can  be  estimated 
across  the  Army  in  accordance  with  Army  distribution  priorities.  This  module  was 
not  directly  exercised  in  VAA  Phase  II. 

(8)  Step  8  -  Results  and  Displays.  Results  are  displayed  in  a  wide  variety  of 
formats  to  meet  the  needs  of  the  sponsor. 

c.  As  currently  implemented,  five  of  the  eight  VAA  modules  have  been  fully  used 
in  conducting  the  VAA  Phase  II  analysis  and  follow-on  quick  reaction  analyses 
(QRAs).  Of  the  remaining  three  modules,  issue  clarification  and  results/display  have 
been  used  in  these  studies,  but  no  specific  tools  or  models  have  been  developed  for 
their  implementation.  Only  the  reso  '.rce  allocation  module  has  never  been  used.  All 
of  the  methodological  refinements  have  concentrated  on  those  five  modules  which 
actually  implement  the  VAA  methodology  for  the  RDA  appropriations.  The  VAA 
Phase  n  study  directive  also  listed  the  implementation  of  the  VAA  methodology  on 
the  METAPHOR  computer  systems  at  HQDA  as  a  requirement.  The  study  team  has 
had  limited  success  in  accomplishing  this  task. 
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2-6.  METHODOLOGICAL  REFINEMENTS 

a.  As  an  outgrowth  of  the  findings  from  the  VAA  Phase  I  Study,  several  methodo¬ 
logical  refinements  were  suggested.  Three  general  areas  requiring  refinement 
included  combat  analysis,  development  of  weights  for  the  coefficients,  and  optimi¬ 
zation.  Although  improvements  to  the  costing  methodology  did  occur,  the  basic 
structure  and  approach  to  the  cost  analysis  did  not  change.  Each  of  the  three  areas  is 
discussed  in  more  detail  in  the  following  paragraphs. 

b.  The  refinements  to  the  combat  analysis  came  in  two  areas.  These  areas 
included  refinements  to  the  combat  simulation,  CORE  AN,  in  terms  of  the  systems 
considered,  and  the  development  of  a  method  to  handle  large  mixes  of  systems 
without  conducting  full-scale  simulations.  First,  the  quality  and  quantity  of  the 
combat  simulations  were  greatly  improved.  The  original  work  completed  in  VAA 
Phase  I  was  very  limited  in  scope.  Only  nine  systems  were  simulated  in  the  first 
phase  in  only  one  scenario.  VAA  Phase  11  increased  both  the  number  of  systems  (to 
41),  and  the  scenarios  included  were  increased  to  3.  The  increase  in  the  number  of 
systems  represented  not  only  a  numerical  difference,  but  also  a  difference  in  the 
variety  of  mission  areas  to  be  considered.  This  increase  was  intended  to  represent  the 
full  spectrum  of  issues  the  methodology  would  be  required  to  handle.  The  other 
improvement  involved  the  development  of  a  response  surface  methodology  to  handle 
the  increased  number  of  systems  for  consideration.  It  became  obvious  that  the  larger 
number  of  potential  mixes  of  systems  could  not  be  individually  simulated  in  separate 
runs  as  was  done  in  the  VAA  Phase  I  work.  The  overhead  and  time  required  to  do  so 
was  too  great  to  be  feasible  within  given  time  constraints.  An  experimental  design 
was  introduced  which  allowed  the  building  of  a  response  surface  that  could  be  used  to 
estimate  the  combat  contribution  of  each  system  being  considered  and  predict  the 
combat  outcome  that  would  be  obtained  if  a  particular  mix  was  played.  The  net  effect 
of  both  of  these  refinements  was  to  make  the  VAA  approach  more  useful  in  dealing 
with  real-world  problems  and  issues. 

c.  The  second  major  area  of  refinement  dealt  with  the  building  of  the  weights  for 
the  effectiveness  criteria.  The  basic  survey  approach  remained  unchanged  from  the 
procedure  in  the  Phase  I  Study.  The  most  significant  refinement  to  this  section  of  the 
methodology  came  in  the  manner  in  which  TOPSIS  was  used  to  build  the  weights. 
Saaty's  analytical  hierarchy  process  (AHP)  was  used  to  develop  the  framework  for 
combining  the  measures  of  effectiveness  into  a  single  value.  However,  at  the  lowest 
level  of  the  hierarchy,  TOPSIS  was  used  to  combine  the  criteria  in  order  to  apply  the 
higher  level  (higher  in  the  analytical  hierarchy)  weights.  This  area  has  been,  and 
remains,  one  of  the  areas  requiring  further  research  and  refinement. 

d.  The  refinement  of  the  optimization  was  probably  the  most  significant  area  of 
methodological  improvement.  In  VAA  Phase  T,  a  very  simplified  linear  programming 
formulation  was  used  in  order  to  prove  the  concept  of  VAA.  The  VAA  Phase  I  find¬ 
ings  clearly  indicated  the  need  to  consider  cost/quantity  relationships,  production 
line  capabilities,  and  greater  detail  in  regard  to  force  structure  constraints.  The 
VALOR  formulation  for  the  Phase  II  Study  incorporated  a  piecewise  linear  approxi¬ 
mation  of  costs  with  respect  to  quantities.  The  ability  to  manipulate  these  relation¬ 
ships  within  the  optimization  algorithm  allowed  the  model  to  reflect  the  effects  of 
economies  of  scale  on  the  procurement  recommendations,  ensuring  a  more  realistic 
and  executable  program.  The  ability  to  handle  production  line  criteria  was  not 
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enhanced  greatly  during  the  VAA  Phase  H  Study.  However,  the  model  did  explicitly 
handle  production  line  consideration  to  include  production  years  and  campaigns.  The 
ability  to  link  systems  together  because  of  force  structure  and  operational  considera¬ 
tions  was  important  to  ensure  rational  and  executable  programs.  The  recommended 
strategies  developed  from  this  formulation  obtained  a  high  level  acceptance  among 
the  decisionmakers  because  of  the  constraints  listed  above. 

2-7.  SUMMARY.  The  VAA  Phase  I  Study  provided  a  solid  foundation  for  the 
conduct  of  weapon  systems  tradeoffs.  The  Phase  II  Study  was  able  to  build  upon  that 
foundation.  The  improvement  was  especially  true  in  the  areas  of  RSM  for  predicting 
combat  results,  the  extension  of  the  Life  Cycle  Cost  Model  to  include  the  41  systems 
of  Phase  11,  and  the  formulation  of  VALOR  to  incorporate  learning  curve  costing  and 
system  linkages. 
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CHAPTERS 
ISSUE  DEFINITION 


3-1.  INTRODUCTION 

a.  The  purpose  of  this  chapter  is  twofold.  First,  the  Issue  Definition  module  is 
explained  and  its  evolution  during  the  VAA  Phase  IT  effort  is  discussed.  Second,  the 
factors  that  established  the  broad  context  of  the  VAA  Phase  n  Study  and  the 
rationale  behind  them  are  discussed.  These  factors  include  timeframes,  scenarios, 
list  of  major  item  systems  to  be  analyzed,  combat  MOE  to  be  used,  and  implicit  factors 
to  be  utilized. 

b.  During  the  conduct  of  the  VAA  Phase  11  Study,  it  was  found  that  previous 
thoughts  and  assumptions  regarding  how  issues  would  be  provided  by  the  study 
sponsors,  particularly  those  that  were  described  in  the  VAA  Phase  I  Study  Report, 
would  have  to  be  modified.  The  primary  assumption  was  that  issues  would  be  fairly 
well  defined  and  that  some  set  procedure  could  be  used  to  structure  them  into  the 
form  needed  to  perform  value  added  analysis.  This  assumption  turned  out  to  be  false. 
Typically,  issues  were  stated  in  an  ambiguous  manner,  and  a  great  deal  of  nego¬ 
tiation  and  iteration  were  needed  to  achieve  the  necessary  form.  It  was  also  found 
that  issue  definition  was  an  ongoing  process  rather  than  one  that  would  be  completed 
early  in  the  study.  The  reason  for  this  effect  was  that,  as  the  study  progressed,  the 
sponsors  and  the  study  team  became  more  comfortable  with  using  the  VAAC.  Often, 
when  one  question  was  answered,  others  were  generated,  and  a  feedback  loop  was 
established. 

c.  Issue  definition  can  be  categorized  into  two  distinct,  but  related,  pieces.  The 
first  revolves  around  establishing  the  broad  context  for  the  study.  This  context  is 
developed  by  defining  the  factors  listed  in  paragraph  3-la  above  and  is  accomplished 
at  the  beginning  of  the  study.  This  process  is  described  in  detail  in  this  chapter.  The 
second  piece  involves  framing  in  specific  terms  the  questions  regarding  individual 
systems  and  groups  of  systems  so  that  recommendations  can  be  made  about  tradeoffs 
and  mixes  of  these  systems  to  be  procured.  These  questions  were  answered  in  a  series 
of  QRAs  and  are  summarized  in  Chapter  9. 

d.  The  remainder  of  Chapter  3  is  structured  as  follows: 

•  Timeframes 

•  Scenarios 

•  Major  Item  Systems  List 

•  Explicit  MOE 

•  Implicit  Factors 

•  Summary 
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3-2.  TIMEFRAMES 

a.  As  mentioned  previously,  the  VAA  Phase  11  Study  was  to  analyze  the 
procurement  of  equipment  in  support  of  the  construction  of  the  FY  94  -  FY  99  POM. 
Thus,  these  years  were  of  primary  importance  during  the  study.  Also  of  importance 
were  the  years  of  the  Extended  Planning  Period  (EPP)  which  are  FY  00  -  FY  08. 
Often,  data  describing  the  developmental  systems  that  were  candidates  for 
procurement  during  the  EPP,  including  cost,  effectiveness,  and  production  data,  were 
not  as  reliable  as  that  describing  systems  to  be  procured  in  the  POM  years. 
Nevertheless,  extreme  care  was  exercised  to  ensure  the  best  data  available  was  used. 

b.  Different  systems  perform  better  or  worse  depending  on  the  level  of 
modernization  of  the  enemy  and  on  their  interaction  with  different  friendly  systems. 
These  factors  are  a  function  of  the  timeframe,  so  the  systems  must  be  evaluated  over 
the  range  of  years  under  consideration.  Unfortunately,  the  available  resources  were 
inadequate  to  perform  the  number  of  combat  simulation  runs  that  would  be  needed  to 
evaluate  every  system  in  every  year  that  it  could  be  in  the  force.  As  such,  the 
decision  was  made  to  choose  a  representative  sample  of  years  and  simulate  only 
those.  These  years  were  FY  96  which  represented  the  near-term,  FY  01  which 
represented  the  mid-term,  and  FY  08,  which  represented  the  far-term.  Note  that  the 
assumption  is  made  that  equipment  procured  will  not  be  available  for  fielding  for  2 
years.  Thus,  the  mid-term  timeframe  represents  the  first  year  that  all  equipment 
procured  in  the  POM  years  is  available  in  the  force.  Note  also  that  threat  data  was 
not  available  for  the  FY  08  timeframe.  The  latest  approved  threat  data  was  for  FY 
04.  Consequently,  an  FY  04  threat  was  used  for  the  far-term  scenarios. 

3-3.  SCENARIOS 

a.  The  original  plan  for  VAA  Phase  11  called  for  the  following  three  scenarios  to 
be  used:  Conventional  Forces  Europe,  Southwest  Asia,  and  Northeast  Asia,  Each  of 
these  scenarios  was  to  be  portrayed  in  each  of  the  three  timeframes. 

b.  Subsequent  to  the  decision  regarding  the  scenarios,  the  threat  posed  by  the 
Soviet  Union  in  T^urope  was  greatly  reduced,  resulting  in  the  lessening  of  the 
importance  of  the  CFE  scenario.  The  decision  was  made,  however,  to  continue  to 
include  the  CFE  scenario  in  the  analysis  since  the  scenario  represented  a  situation  in 
which  US  forces  faced  a  high-tech  enemy.  The  use  of  this  scenario  was  consistent 
with  the  concept  of  capability-based  requirements. 

c.  Whereas  data  existed  to  use  as  a  starting  point  for  building  the  SWA  and  CFE 
scenarios  for  the  combat  simulation  model,  none  existed  for  the  NEA  scenario.  As  a 
result,  data  had  to  be  assembled  from  scratch,  greatly  increasing  the  time  required  to 
perform  the  simulation  of  this  scenario.  It  became  evident  that  finishing  these  runs 
in  time  for  use  in  the  analysis  was  impossible.  As  a  result,  this  part  of  the  effort  was 
abandoned  so  that  more  resources  could  be  applied  to  the  other  scenarios. 

d.  The  result  of  the  above  was  that  combat  results  were  available  for  CFE  and 
SWA  only.  These  results  were  later  referred  to  as  the  Stressful  Scenario  and  the 
Contingency  Scenario,  respectively. 
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3-4.  MAJOR  ITEM  SYSTEMS  LIST.  The  decision  regarding  which  systems 
should  be  included  in  the  VAA  Phase  11  Study  was  made  jointly  by  the  sponsors  and 
the  study  team.  The  criteria  used  to  decide  whether  a  system  should  be  included  was 
the  team's  ability  to  model  the  system,  the  existence  of  a  system  requirements 
document,  and  the  determination  that  the  system  represented  a  major  acquisition  in 
terms  of  the  total  dollars  associated  with  the  system's  program.  The  "first  cut”  of  the 
original  119,  based  on  the  previously  mentioned  criteria,  resulted  in  a  list  of  50  major 
item  systems.  Each  of  the  systems  on  this  "first  cut"  list  were  modeled  in  CORE  AN. 
As  the  team  gained  experience  with  particular  systems  and  as  decisions  were  made 
by  the  Army  leadership,  several  systems  were  dropped  from  consideration.  The  final 
list  consisted  of  41  systems.  A  list  of  the  systems  included  in  the  VAA  Phase  11  Study 
is  provided  in  Chapter  4,  together  with  the  timeframes  in  which  they  are  available 
for  fielding. 

3-5.  EXPLICIT  MOE.  In  the  Phase  I  efTort,  three  explicit,  or  combat,  MOE  were 
used.  These  were  Blue  force  surviving  (BPS),  Red  force  movement  (RFM),  and 
correlation  of  forces  and  means  (COFM).  In  addition,  loss  exchange  ratio  (LER)  was 
calculated  but  was  not  used.  Subsequent  analysis  revealed  that  all  of  these  MOE 
except  BFS  were  highly  correlated,  indicating  that  they  were  measurements  of  the 
same  criteria.  As  the  result  of  this  analysis,  the  decision  was  made  to  reduce  the 
number  of  explicit  MOE  to  two,  one  measuring  the  lethality  of  the  force,  and  the 
other  measuring  the  survivability  of  the  force.  Force  exchange  ratio  (FER)  and  BFS 
were  chosen  to  fill  these  respective  roles.  This  reduction  in  the  number  of  explicit 
MOE  had  the  additional  benefit  of  reducing  the  number  of  pairwise  comparisons 
needed  in  the  Senior  Leader  Survey.  Chapter  5  describes  this  survey  and  gives  its 
results. 

3-6.  IMPLICIT  FACTORS.  As  was  the  case  with  the  explicit  MOE,  the  number  of 
implicit  factors  used  was  reduced.  Where  Phase  I  utilized  11  implicit  factors,  6  were 
used  in  Phase  n.  The  following  are  the  implicit  factors  that  were  used.  These  factors 
are  defined  in  Chapter  5,  and  they  represent  an  attempt  to  eliminate  duplication  and 
factors  that  were  not  considered  important  in  Phase  I. 

•  Political  risk, 

•  Programmatics, 

•  Asset  versatility  and  deployability, 

•  Operation  and  technical  risk, 

•  Criticality  of  need  with  respect  to  existing  capabilities,  and 

•  Impact  on  sustainability  with  respect  to  combat. 

3-7.  SUMMARY.  The  process  of  setting  the  modeling  and  context  of  the  Value 
Added  Analysis  is  undoubtedly  the  most  important  step  in  the  VAA  methodology. 

The  results  have  a  long-term  effect  on  the  capability  to  provide  timely  and  useful 
analysis  in  the  POM  building  efTort.  As  such,  extreme  care  must  be  taken  to 
thoroughly  think  through  all  the  ramifications  of  the  decisions  made  in  this  module. 
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CHAPTER 4 

EXPLICIT  MEASURES  OF  VALUE 


4-1.  COMBAT  EFFECTIVENESS 

a.  Introduction.  The  purpose  of  this  chapter  is  to  explain  the  development  of  the 
combat  context  for  Army  Program  Value  Added  Analysis  94-99  (VAA  94-99),  to 
describe  the  models  and  tools  used  in  the  development  of  the  measures  of  effective¬ 
ness,  and  to  present  the  VAA  Phase  II  scenarios  and  the  results  of  the  simulation 
runs. 

b.  Development  of  Combat  Context 

(1)  Guidance.  Unlike  VAA  Phase  I,  where  the  TRADOC-approved  scenarios 
were  used,  the  scenarios  in  the  Total  Army  Analysis,  92-99  (TAA-99)  were  used  in 
VAA  Phase  H;  these  scenarios  adhere  to  Defense  Guidance.  Since  the  TAA-99 
scenarios  were  used  for  theater-level  simulations  in  the  TAA-99  Study,  the  locations 
and  strengths  of  units  required  to  implement  the  corps-level  scenarios  were  readily 
available.  Other  modifications  were  dictated  by  the  unique  requirements  of  the  VAA 
Phase  n  cost-benefit  analysis. 

(2)  Requirements  for  VAA  Phase  II 

(a)  The  following  are  the  scenario  requirements  for  VAA  Phase  II.  First, 
since  VAA  is  an  incremental  analysis,  the  base  case  items  did  not  require  combat 
effectiveness  measures.  Only  the  VAA  candidate  systems  which  are  portrayed  in  the 
excursions  need  to  be  measured  with  respect  to  their  contribution  to  combat  effective¬ 
ness.  Next,  situations  must  be  varied  by  the  terrain,  postures,  missions,  and  threats. 
Third,  the  scenarios  must  be  based  upon  approved  Army  doctrine  and  related  to  Total 
Army  Analysis  (TAA).  Finally,  all  major  weapon  system  groups  must  be  exercised,  to 
include  system  substitutions  and  comparisons  and  to  stress  systems,  not  forces. 

(b)  Certain  modifications  to  scenarios  and  indeed  to  doctrine  are  required  to 
enable  a  cost-benefit  type  analysis  to  be  performed.  For  example,  doctrine  niay  call 
for  a  flanking  maneuver  which  bypasses  the  enemy.  However,  if  this  tactic  is 
employed,  some  weapon  systems  may  not  become  engaged,  and  these  systems’ 
effectiveness  cannot  be  measured.  Therefore,  although  doctrine  may  call  for  a 
bypassing  maneuver,  an  attack  into  the  enemy  may  be  executed  to  ensure  that  the 
systems  of  interest  are  well  exercised.  Note  that  the  values  of  the  MOE  can  be 
altered  by  chanmng  the  intensity  of  combat  the  system  endures.  Care  must  be  taken 
to  ensure  that  all  tactics  employed  in  the  combat  simulation  are  reasonable. 
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(c)  The  following  are  the  modifications  to  the  TAA-99  scenarios  that  were 
incorporated  when  building  VAA  Phase  II  scenarios.  First,  only  US  assets  were 
considered,  since  VAA  is  concerned  only  with  US  Army  programs,  and  assets  were 
limited  due  to  the  current  combat  model  size  constraints.  Next,  the  focus  of  combat 
effectiveness  measures  was  on  modeling  one  US  corps.  This  level  of  resolution  was 
found  to  be  most  appropriate,  since  systems  could  be  portrayed  in  adequate  detail  to 
ascertain  their  individual  contributions  to  the  outcome  of  battle,  which  would  not 
have  been  the  case  using  existing  theater-level  models;  but  it  did  include  sufTicient 
command  and  control  and  logistics  aspects  that  would  not  have  been  available  if  the 
division  level  of  resolution  was  used.  AirLand  Operations  doctrine  was  used  when¬ 
ever  possible.  Finally,  the  Army  of  Excellence  force  structure  was  used. 

(d)  The  following  scenarios  were  planned  to  be  used  for  the  simulations  to 
cover  the  programming  period:  SWA,  CFE,  and  NEA.  The  period  of  interest  in  VAA 
Phase  n  is  FY  94-99,  the  POM  period,  and  FY  00-08,  the  EPP.  A  lag  time  of  2  years 
between  systems  procurement  and  fielding  was  assumed.  For  example,  systems 
procured  in  FY  94  would  be  available  to  fight  an  enemy  in  FY  96.  The  EPP  actually 
extends  to  FY  08,  requiring  a  simulated  battle  in  FY  10.  However,  threat  data  was 
not  available  for  that  timeframe,  so  the  available  FY  04  data  was  used.  For  each 
scenario,  simulated  battles  were  conducted  representing  FY  96,  the  beginning  of  the 
POM  period;  FY  01,  the  end  of  the  POM  period;  and  FY  08,  the  end  of  the  EPP.  A 
complete  description  of  the  scenarios  is  given  in  Volume  H,  Appendix  F,  published 
separately. 

c.  Analytical  Models  and  Tools 

(1)  Corps  Battle  Analyzer  (CORBAN).  The  version  of  CORBAN,  as  used  in 
VAA  Phase  II,  is  a  deterministic  force-on-force  simulation  of  corps-level  battles  at 
battalion-level  resolution.  Figure  4-1  lists  the  input  and  output  parameters  of  the 
model.  For  a  more  complete  description  of  CORBAN,  see  the  CORBAN  User’s  Guide. 
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Input 


•  Scenario  and  plans 

Task  organization 

Situation 

Mission 

Execution 

Service  support 

Command  and  signal 

•  TOE 

•  SOP  for  operations 

•  Asset  characteristics 

Ex  SSPk  vs  1 2  tgt  types 

Fire  rate 

Ranges 

Ammo  type  use 
Fuel  use 
Crew  levels 
Vulnerabilities 
RAM  factors 

•  Terrain 

•  Search  patterns 

•  Sensor  classes 


Process 


3  cycles  in  each  10- 
mmute  time  intervaf; 

Shoot 

Move 

Recover 


Output 


•  Measure  of  effectiveness 

Correlation  of  forces 
Force  ratios 
Loss  exchange  ratios 
Fractional  exchange  'atios 
System  effectiveness  ratios 
Killer-victim  tables 
Effective  battalions  remaining 
Center  of  mass  movement 
Mission  accomplishment 
Front-line  trace 
Plots  of  unit  locations 

•  Debugging 

Decision  traces 
Communications  traces 
Unit  status 
Attrition 

Combat  processes 
Logistic  traces 
Movement  traces 

•  Maps  and  Overlays 


Figure  4-1.  The  CORBAN  Model 


d.  Implementation.  During  VAA  Phase  II,  the  CORBAN  Model  was  transferred 
from  the  VAX  computer  and  was  run  on  a  network  of  SUN  work  stations.  This 
transfer  allowed  several  CORBAN  runs  to  be  made  simultaneously,  which  shortened 
the  time  needed  to  complete  sets  of  runs.  This,  together  with  the  development  of  the 
deterministic  version  of  CORBAN,  which  requires  only  one  replication  per  excursion, 
reduced  the  amount  of  time  needed  to  complete  the  combat  simulations.  Otherwise, 
it  would  have  been  impossible  to  have  the  combat  results  by  the  time  they  were 
needed.  Even  so,  the  decision  was  made  to  eliminate  the  series  of  NEA  scenarios  and 
use  only  one  Contingency  and  one  Stressful  Scenario.  This  decision  was  made 
because  NEA  required  an  entirely  new  base  case  to  be  developed,  and  not  enough 
time  was  available  to  accomplish  this  task. 
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e.  Systems  Modeled 

(1)  The  following  is  a  list  of  VAA  weapon  systems  that  were  modeled  in 
CORBAN  and  for  which  results  were  obtained. 

Year  Acronym  Name  of  system 

1996  AAWS-M  Advanced  Antitank  Weapons  System  -  Medium 

1996  AD  ATS  Air  Defense  Antitank  System 

1996  AdvQUICKFIX  Advanced  Quickfix  Electronic  Warfare  System 

1996  ARV  Armored  Recovery  Vehicle 

1996  ATAS  .r-to-air  STINGER 

1996  AVENGER  AVENGER 

1996  FMTV  2.5/5  Family  of  Medium  Tactical  Vehicles  2.5  Ton/5  Ton 

1996  GBCS-L/H  Ground  Based  Common  Sensor  -  Light/Heavy 

1996  HAB  Heavy  Assault  Bridge 

1996  HET  Heavy  Equipment  Transporter 

1996  JSTARS  Joint  Surveillance  Target  Attack  Radar  System 

1996  M1A2  Ml  A2  Abrams  Main  Battle  Tank 

1996  MLRSSADARM  Multiple  Launch  Rocket  System  Sense  and  Destroy 

Armor 

1996  PLS  Palleti2ed  Loading  System 

1996  TOW  2B  Tube-launched,  Optically  tracked,  Wire  Guided  Missile 

2B 

1996  VEMASID  Vehicle  Magnetic  Signal  Duplicator 

1996  VOL  Air  VOLCANO  Multiple  Delivery  Mine  System  -  Air 

1996  VOL  Gnd  VOLCANO  Multiple  Delivery  Mine  System  -  Ground 

1996  WAM  Wide  Area  Mine 

2001  155  ER  155mm  Extended  Range  Round 

2001  155  LtWt  155mm  Light  Weight  Howitzer 

2001  155  TGP  155nim  Terminally  Guided  Projectile 

2001  Ad  V  Fire  finder  Advanced  Firefinder 

2001  AGS  Armored  Gun  System 

2001  AH-64  LB  Apache  Attack  Helicopter  with  LONGBOW 

2001  AMS-H  Advanced  Missile  System  -  Heavy 

2001  ATACMS  II  Army  Tactical  Missile  System  Block  II 

2001  GBS  Ground  Based  Sensor 

2001  LOSAT  Line  of  Sight  Antitank 

2001  MLRSTGW  Multiple  Launch  Rocket  System  Terminally  Guided 

Weapon 

2001  NLOS-AT  Nonline  of  Sight- Antitank 

2001  RAH-66  Comanche  Reconnaissance  Attack  Helicopter 

2001  STINGRAY  STINGRAY  Combat  Protection  System 

2004  AdvSEMA  Advanced  Special  Electronic  Mission  Aircraft 

2004  AFAS  Advanced  Field  Artillery  System 

2004  BLOCK  III  Block  III  Main  Battle  Tank 

2004  CMV  Combat  Mobility  Vehicle 

2004  FARV-A  Future  Armored  Resupply  Vehicle  -  Artillery 

2004  FIFV  Future  Infantry  Fighting  Vehicle 

2004  RAH-66  LB  Comanche  Reconnaissance  Attack  Helicopter  with 

LONGBOW 
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(2)  A  description  of  the  major  aspects  of  each  weapon  system,  as  modeled  in 
CORBAN,  is  found  in  Appendix  F,  Volume  II,  of  this  report. 

(3)  Note  that  several  additional  systems  were  included  in  the  original  list. 
However,  they  were  ultimately  omitted  because  they  were  found  to  be  difRcult  to 
model  or  were  eliminated  from  procurement  consideration  prior  to  the  beginning  of 
the  study. 

f.  Simulation  Results 

(1)  Results  by  individual  run  have  no  meaning,  since  each  run  is  an 
independent  mix  of  systems.  One  run  cannot  be  compared  meaningfully  with 
another.  The  run  results  must  be  processed  by  the  RSM  to  obtain  system  results;  see 
paragraph  4-7. 

g.  Lessons  Learned  and  Work  for  Phase  III 

(1)  Probably  the  most  important  lesson  learned  is  to  begin  modeling  and  testing 
the  systems  early.  There  is  no  doubt  that  some  anomalous  results  were  caused  by 
unusual  adverse  synergistic  effects.  Because  each  run  has  as  many  as  40  new 
systems,  it  is  impossible  to  identify  causal  effects  with  surety.  Verification  of  input 
data  allows  the  identification  of  data  and  modeling  errors  in  some  cases,  and  in 
others,  a  better  way  to  model  certain  systems  the  system  in  question  is  required. 

With  enough  time,  a  more  scientific  approach  could  be  undertaken.  Each  system 
could  be  modeled  and  tested  by  itself  in  a  full-up  scenario,  then  placed  in  a  combined 
arms  context  to  measure  positive  and  negative  effects.  Finall}',  the  approved  scenario 
could  be  used  to  test  each  system.  It  would  be  impossible  to  run  each  system  with 
every  other  system  individually,  but  one  model  of  the  final  scenario  could  be  set 
aside,  and  each  system  could  be  added  and  run  one  by  one.  Each  run  could  be 
compared  with  preceding  runs  to  identify  negative  impacts.  This  procedure  would 
reduce  the  incidence  of  data  and  modeling  errors  in  the  final  record  runs  of  the  model. 

(2)  No  doubt  the  second  most  import  lesson  is  to  point  out  the  need  to  put  all  of 
the  records  necessary  to  model  systems  into  an  input  data  base  management  system. 
The  data  base  management  system's  own  language  could  be  used  to  perform  some 
simple  data  checks.  Software  could  then  be  written  to  automatically  build  the 
CORBAN  run  files  from  the  data  base  of  systems.  Other  programs  could  be  written  to 
build  all  the  runs  necessary  to  execute  an  experiment  by  reading  in  an  experimental 
design  matrix. 

4-2.  MEASURES  OF  EFFECTIVENESS (MOK).  The  following  MOE  were  used 
to  measure  systems’  contributions  to  combat  effectiveness. 

a.  Fractional  Exchange  Ratio  (FER)  -  measures  the  fractional  Red  losses 
compared  to  fractional  Blue  forces.  FER  is  computed  as  follows: 


Red  Systt'm  l^onfies/Red  Sy^itema  Started 

pi*  ^  _  -r 

Blue  System  flosses/ Blue  Systems  Sta  rted 

The  intent  was  for  FER  to  measure  how  well  Red  is  killed  by  Blue. 


(1) 
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b.  Blue  Force  Surviving  (RFS)  -  determines  the  percentage  of  Blue  strength 
remaining.  Blue  strength  is  determined  by  the  number  of  items  times  their  weight, 
compared  to  the  beginning  strength  of  the  Blue  Forces.  Blue  strength  is  computed  a:> 
follows: 


Blue  Strength  —  ^  ^Number  of  Blue  system  surviving  ^  *  Systerns  weight  '2) 

1=  l 

where  i  =  the  system  number. 

The  system  weight  is  the  effectiveness  index  parameter  from  CORE  AN  (see  the 
CORJBAN  User’s  Guide  for  details).  This  MOK  was  used  to  measure  the  survivability 
force  as  different  combinations  of  systems  were  employed. 

4-3.  INTRODUCTION  TO  RESPONSE  SL  RFACK  METHODOLOGY  (RSM). 

A  detailed  explanation  of  RSM  and  the  use  of  experimental  design  can  be  found  in  the 
CAA  Technical  Paper,  Experimental  Design  with  Combat  Models  (XD  COMBAT), 
CAA-TP-92-9. 

a.  The  requirement  exists  to  determine  the  combat  contribution  of  each  weapon 
system  to  the  outcome  of  the  battle  as  measured  by  thp  explicit  MOE.  A  common 
method  of  accomplishing  this  task  is  to  establish  a  baseline  case  which  includes  no 
new  system,  then  to  add  or  substitute  each  new  weapeo  system  one  at  a  time, 
measuring  the  changes  in  combat  elTectiveness.  Alternatively,  a  base  case  can  be 
established  which  includes  all  new  weapon  systems,  and  then  deleting  one  in  each 
excursion,  measuring  the  chunges  in  simulation  out<-''m*'  Tht-’se  changes  from  the 
baseline  case  measure  th  '  contribution  of  a  weapon  sy^tetn  In  VAA  Phase  I,  the 
latter  technique  was  used.  Whi  le  this  method  measur.  .  mtribution  of  each 
individual  system,  it  does  nof  allow  for  the  determination  of  the  combined  effect  of 
weapon  systems  mixes,  i.e.,  if  an  attack  helicopter  raises  the  value  of  an  MOE  by  "x" 
and  a  tank  raises  the  value  hy  "y"  as  individual  substitutions,  then  it  is  not  true  that 
if  both  systems  are  present,  the  resulting  improvement  would  be  "x  -I-  y." 

b.  The  ideal  method  for  df  termlning  ‘he  optimal  niix  of  new  systems  would  be  to 
explore  all  possible  combinations.  This  niethod  would  find  the  combination  of 
systems  that  yields  the  greatest  increase  in  the  MOE  values.  While  this  method  is 
practical  for  situations  where  the  number  of  systems  to  be  ev  aluated  is  small,  the 
number  of  combinations  grows  quickly  as  the  number  of  such  systems  increases.  P'or 
3  systems,  there  are  23  or  8  combinations.  If  one  had  to  explore  every  combination  of 
40  different  systems,  the  number  of  potential  runs  would  be  2^^,  or  109.9  billion  runs. 

c.  RSM  represents  a  compromi'-..  between  the  proce,>.'  of  replacing  weapons  one  at 
a  time  and  the  ideal  solution  of  exarnlfiing  ery  combi n.i lion.  This  compromise  is  a 
fractional  design,  meaning  that  a  .spot  i  fie  subset  of  the  combinations  is  used.  The 
subset  of  combinations  of  systems  b  be  examined  is  determined  using  a  particular 
experimental  design,  "^he  resulting  "design  matrix"  varies  the  combinations  in  an 
efficient  manner  so  that  a  general  linear  mudel  can  be  built  to  forecast  the  efforts  if 
the  systems  with  respect  to  the  outpuLs.  A  set  of  coefficients  is  computed  which  is  tlic 
mean  or  average  contribution  given  the  new  weapon  system.  These  coefficients  car' 
then  be  used  in  an  additive  estimate. 
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4-  i.  USE  OF  RSM 

a.  "RSM  comprises  a  group  of  statistical  techniques  for  empirical  model  building 
and  model  exploitation.  By  careful  design  and  analysis  of  experiments,  it  seeks  to 
relate  a  'response,’  or  output  variable  to  the  levels  of  a  number  of 'predictors,’  or  input 
variables,  that  affect  it.”  As  illustrated  in  Figure  4  2,  the  output  values  from 
CORBAN,  namely,  the  specified  MOE,  are  functions  of  the  various  input  values.  The 
purpose  of  RSM  is  to  identify  these  functional  relationships. 


Inputs  Outputs 

Blue  systems 
Red  systems 
Terrain 
Seer,  'rio 


Figure  4-2,  CORBAN  with  All  the  Inputs 


b.  In  VAA,  the  Red  systems,  the  terrain,  and  the  scenario  input  variables  are  held 
constant  and  the  Blue  systems  that  will  be  studied  are  varied.  Thus,  the  inputs  into 
the  black  box  are  reduced  to  a  more  manageable  si7f'  This  reduction  is  as  illustrated 
in  Figure  4-3. 


Inputs 
Xi  - 

Blue  ^2  - 
systems  _ 

X4  - 


PTgure4-3.  Controlling  onl^  the  Desired 
Inputs 

c.  A  general  linear  model  is  used  t"  estimate  an  output  or  dependent  variable,  Y, 
whose  mean  is  a  function  of  one  or  more  indepr  ndent  variables  (xi,  X2,  etc.).  The 
general  linear  model  has  the  form: 


Y  =  h  +  h^x  ■>-  h  X  ^  />  ,,1 ,  "  ^  h  X 

j  I  I  2  2  ^ 


Outputs 
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where  bi  is  the  individual  average  response  of  the  independent  variables  '^n  Y.  In 
matrix  form,  the  general  linear  model  appears  as: 

Y  =  XB  (4) 

In  the  matrix  format,  Y  is  a  vector  of  the  model  results  for  a  particular  MOPI.  Each 
element  of  the  vector  is  the  value  corresponding  to  a  particular  excursion.  B 
represents  the  vector  of  bis,  with  bo  representing  the  effect  of  the  base  case  weapcjns, 
and  bi  through  br  are  the  effects  of  systems  1  through  r  in  the  excursions.  Finally,  X 
is  the  design  matrix  of  binary  independent  variables  whose  construction  is  descr.bed 
below.  This  mathematical  model  isolates  only  the  presence  of  main  effects  without 
considering  any  interaction  efTects. 

d.  Plackett-Burman  designs  are  useful  when  the  problem  of  determining  the 
main  effects  with  maximum  precision  is  reduced  to  a  combinatorial  problem.  They 
are  useful  when  the  problem  has  only  two  level  factors,  i  e  ,  when  there  are  low  and 
high  variable  settings  or  binary  (0,  1)  variables.  In  VA  A,  the  systems  being 
considered  for  procurement  are  the  factors.  A  "1"  represents  the  presence,  in  the 
appropriate  quantities  for  the  modeled  force  stiucture,  in  the  excursion.  A  "0” 
indicates  that  none  of  the  items  are  played  in  that  excursion.  Plackett-Burman's 
method  specifies  the  construction  of  the  design  matrix,  X.  Th’s  matrix  represents  a 
map  of  all  of  the  independent  variables'  values  for  each  cnmjiuter  run.  Each  row 
corresponds  to  a  specific  computer  ruti  and  each  column  coi  f  esj  nds  to  a  different 
factor. 

e.  To  illustrate  this  coding  scheme,  consider  the  following  tW')  cases.  The  first 
case  involves  a  new  system  replacing  an  existing  o  An  example  of  this  case 
would  be  the  Advanced  Antitank  Weapons  System  \b<hum  (AAWS-M)  replacing  the 
DRAGON.  In  excursions  where  soldiers  are  equipped  with  the  AAWS-M,  a  1  would 
be  entered  in  the  design  matrix.  In  excursions  where  the  DKAOON  is  used,  a  0  would 
appear. 

f.  The  second  case  involves  a  new'  system  that  does  not  replace  an  existing  system. 
The  Joint  Surveillance  Target  Attack  Radar  System  (JS'l  ARS)  would  be  such  a 
system.  In  this  case,  a  1  would  indicate  the  presence  of  the  new  system,  while  a  0 
would  denote  its  absence. 

g.  Once  the  design  matrix  is  formed,  each  comVral  model  excursi  i  is  performed 
using  the  systems  specified  as  in  the  design;  and  in  turn,  the  outputs  li.e.,  BFS  or 
FPiR)  are  produced  forming  the  Y  vector.  The  coefficients  (elements  of  B)  for  the 
linear  model  are  obta'ned  by  matrix  algebra,  then 

XB  ^  V  (5) 

To  solve  for  B,  there  are  two  options.  If  a  full  Plackett-Burman  matrix  is  used 
without  deleting  any  columns  such  that  X  is  of  full  rank,  then; 


R  X 


(6) 


4-3 


CAA-SR-92-10 


If  there  is  a  need  or  desire  to  use  a  reduced  matrix,  then  use  the  standard  formula  for 
solving  a  general  linear  model: 


B  - 


(7) 


The  above  matrix  manipulation  can  be  done  on  spreadsheets,  IMSL  routines, 
FORTRAN  programs,  SPSS,  BMDP,  or  other  statistical  programs. 

Example:  In  this  example,  the  seven  weapon  systems  were  examined.  These 
systems  are  weapons  1  to  4,  field  artillery  (FA)  weapons  5  and  6,  and  attack 
helicopter  7.  After  setting  up  the  experimental  design,  conduct  a  combat  simulation 
including  those  systems  in  a  row  which  have  a  1  in  the  design  matrix.  The  result 
from  each  combination  is  recorded  as  a  value  for  the  MOE  of  interest.  Since  the  X 
matrix  is  full  rank,  use  equation  4-6. 


X  = 


Design  matrix  MOE 


1 

1 

1 

1 

0 

1 

0 

0 

23 

1 

0 

1 

1 

1 

0 

1 

0 

45 

1 

0 

0 

1 

1 

1 

0 

1 

24 

1 

1 

0 

0 

1 

1 

1 

0 

Y  = 

34 

1 

0 

1 

0 

0 

1 

1 

1 

56 

1 

1 

0 

1 

0 

0 

1 

1 

67 

1 

1 

1 

0 

1 

0 

0 

1 

78 

1 

0 

0 

0 

0 

0 

0 

0 

3 

Run  # 

1 

2 

3 

4 

5 

6 

7 

8 


0 

0 

0 

0 

0.25 

-0.25 

-0.25 

0.25 

0.25 

0.25 

-0.25 

-0.25 

0.25 

0.25 

0.25 

-0.25 

-0.25 

0.25 

0.25 

0.25 

0.25 

-0.25 

0.25 

0.25 

-0.25 

0.25 

-0.25 

0.25 

-0.25 

-0.25 

0.25 

-0.25 

0 

0 

0 

1 

-0.25 

0.25 

0.25 

■0.25 

0.25 

-0.25 

0.25 

-0.25 

-0.25 

0.25 

-0.25 

-0.25 

-0.25 

-0.25 

0.25 

-0.25 

0.25 

-0.25 

-0.25 

-0.25 

0.25 

0.25 

-0.25 

-0.25 

0.25 

0.25 

0.25 

-0.25 

B  =  X  ^  Y 


B  = 


3 

18.5 

18.5 

-3 

8 

-14 


18.5 

30 


Mean  value  of  the  base  case 

Weapon  1 

Weapon  2 

Weapon  3 

Weapon  4 

t  ield  artillery  5 

Field  artillery  6 

Attack  helicopter  7 
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The  equation  for  the  predicted  MOE  based  on  the  above  results  is: 

MOE  =  3  -h  18.5  WPNl  -h  18.5  WPN2  -  3  WPN3  +  8  WPN4  - 14  FA5  +  18.5  FA6  + 

30ATKH7 


where  each  system  is  either  1  or  0. 

4-5.  MEANING  OF  VALUES 

a.  While  the  general  linear  model  is  normally  used  for  fitting  a  regression  line 
among  quantitative  variables,  it  can  be  used  with  qualitative  variables  as  well.  The 
estimator  bo  is  the  measure  of  worth  of  the  base  case  weapons.  The  estimators  bi 
through  br  represent  the  contribution  of  individual  systems  to  the  change  in  the 
corps-level  MOE  (i.e.,  FER  and  BFS).  "These  coefficients  are  often  called  differential 
intercept  coefficients  because  they  reveal  by  how  much  the  value  of  the  ,’ntercept 
term  of  the  category  that  receives  the  value  1  differs  from  the  intercept  coefficient  of 
the  category  that  receives  the  value  0.  The  category  that  receives  the  value  zero  is 
often  referred  to  as  the  base  or  comparison  category." 

b.  Another  advantage  of  the  above  formulation  is  that  the  general  linear  model 
can  be  used  to  evaluate  difTerent  force  packages  or  mixes  of  systems  without 
rerunning  CORBAN.  The  linear  model  can  then  be  used  to  give  an  estimate  of  the 
values  of  FER  and  BFS  for  the  new  combinations  of  systems  without  conducting 
additional  simulation  runs. 

4-6.  PROBLEMS  USING  THIS  TECHNIQUE 

a.  This  procedure  is  not  a  cure  for  input  errors  and  intense  weapon  system 
interactions,  nor  does  it  immediately  help  to  explain  counterintuitive  results. 
Thorough  analysis  is  still  required  to  check  the  answers,  and  then  serious  thought  is 
needed  to  interpret  the  results. 

b.  When  this  methodology  was  first  used  in  the  VAA  Study,  the  discovery  of  input 
errors  was  difficult.  When  the  traditional  method  of  combat  modeling  is  used,  only 
one  system  at  a  time  is  adjusted  from  the  base  case.  The  modelers  can  then  easily 
compare  the  results  from  their  excursions  to  the  base  case  and  determine  the 
differences  and  find  any  errors.  It  is  much  more  difficult  to  find  modeling  errors  in  an 
experimental  design  when  there  are  many  different  new  systems  in  each  run.  Once 
major  errors  are  discovered,  rerunning  the  simulation  may  be  required  to  correct  the 
error. 

c.  A  major  concern  arose  when  systems’  coefficients  were  negative.  Intuition  tells 
the  analyst  that  the  replacement  of  an  old  system  with  a  vastly  superior  system 
should  not  result  in  a  decrease  in  capability.  An  examination  of  the  systems  then 
showed  unnoticed  modeling  data  problems  with  the  negative  systems.  Examples  are 
search  patterns  larger  than  the  arrays  set  up  to  store  them,  firing  rates  set  to  zero,  or 
an  incorrect  portrayal  of  the  weapon.  The  methodology  has  not  yet,  in  its  use  in  VAA, 
produced  a  result  that  was  worse  for  a  clearly  superior  system  when  the  input  data 
for  the  respective  new  and  old  systems  were  correct. 
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d.  To  assist  in  error  checking,  other  summary  or  intermediate  statistics  were 
used  to  find  problems.  An  example  is  the  number  of  losses  for  each  side.  For 
instance,  if  the  FER  for  a  system  is  negative,  the  major  components  of  FER,  the 
number  of  Red  losses,  and  the  number  of  Blue  losses  are  examined.  The  regression 
was  applied  to  these  statistics,  and  interesting  discoveries  could  be  made.  The 
presence  of  some  weapons  was  linked  to  Blue  experiencing  more  losses  during  these 
runs.  Calculations  showed  that  it  was  the  increase  of  Blue  losses  that  made  the 
system's  FER  turn  negative.  This  technique  helped  lead  to  discovering  the  reason 
why  there  were  more  Blue  losses.  An  example  is  when  the  Advanced  Missile  System- 
Heavy  (AMS-H)  and  tube-launched,  optically  tracked,  wire  guided  missile  2  ’  (TOW 
2B)  missiles  killed  less  smaller  Red  weapons  (i.e.,  antitank  guided  missiles  (ATGM.-^  ) 
than  the  TOW  2A  which,  in  turn,  killed  more  Blue  sysLertia.  The  solution  was  to 
model  the  Bradley  chain  gun. 

e.  In  some  cases,  an  interaction  between  two  systems  occurs.  This  interaction  can 
be  a  synergism  between  two  systems  that  greatly  inci  eases  their  ability.  A  pairing  of 
a  long  range  sensor  and  a  long-range  missile  is  an  example  of  a  possible  synergistic 
interaction.  The  interaction  could  also  decrease  the  systems'  MOE  contribution,  such 
as  two  long-range  weapons  that,  when  together,  compete  for  the  same  targets.  A 
solution  to  this  problem  is  called  a  "foldover”  design  that  isolates  the  main  effects  of 
the  interested  systems.  The  cost  of  this  "foldover”  design  is  to  double  the  number  of 
computer  runs.  Other  designs  to  specifically  measure  i  nteractions  are  available,  but 
their  cost  in  the  number  of  computer  runs  is  prohibitive  except  in  problems  with  only 
a  limited  number  of  independent  variables. 

f.  Interpretation  of  combat  results  has  never  been  easy.  Time  and  thought  still 
need  to  be  invested  to  determine  why  a  system  is  drunc  sa  pout  ly.  or  doing  so  well,  and 
then  to  present  the  results  to  the  decisionmakers.  l)«  ■  .'il■ruM.^kers  need  to  under¬ 
stand  the  reasons  for  an  answer,  and  RSM  niay  makt  i  l  mur  e  difficult  for  the  modeler 
to  explain  the  results.  In  this  particular  situatiur.,  a  diila  base  system  is  invaluable  to 
manipulate  the  large  amount  of  data  needed  to  perform  these  analyses.  Explaining 
and  verifying  counterintuitive  results  is  a  difficult  and  tedious  process. 

4-7.  RKSL  l.'I'S  FROM  RSM.  The  following  system  results  were  obtained  for  FKK 
and  BFS  for  each  scenario- year  group  combination.  These  values  are  additive  for 
only  that  particular  scenario/year  MOFi  combination  For  instance,  to  forecast  the 
BFS  in  the  Contingency  96  Scenario  for  a  particular  system  mix,  one  added  the 
intercept  term  and  then  all  other  values  for  specific  VAA  systems. 

a.  Tabl"'  4-1  contains  the  Contingency  96  Scenario  results. 
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Table  4-1.  RSM  Results  for  Contingency  96  Scenario 
(contribution  of  systems  to  MOE) 


Systems 

1996 

2001 

2008 

FER 

BFS 

FER 

BFS 

FER 

BFS 

INTERCEPT 

2.378 

0.750 

2.458 

0.805 

2.842 

0.825 

AAWS-M 

0.275 

0.023 

-0.018 

0.003 

-0.119 

0.000 

ADATS 

0.263 

0.017 

1.779 

0.033 

1.610 

0.023 

ADV  QUICKFIX 

0.200 

0.022 

-0.048 

-0.002 

0.152 

0.009 

ARV 

-0.002 

-0.002 

0.100 

0.004 

-0.235 

-0.004 

ATAS 

0.170 

0.011 

0.351 

0.008 

-0.079 

-0.001 

AVENGER* 

NA 

NA 

NA 

NA 

NA 

NA 

FMTV  2.5/5 

-0.148 

-0.009 

-0.020 

0.002 

0.332 

0.011 

GBCS-L/H 

0.068 

0.004 

-0.015 

0.001 

0.447 

0.012 

HAB 

0.159 

0.006 

0.341 

0.011 

0.128 

0.004 

HET 

-0.037 

0.002 

0.024 

0.006 

0.175 

0.010 

JSTARS 

0.193 

0.014 

0.029 

0.001 

0.025 

0.000 

M1A2 

-0.058 

0.000 

0.500 

0.014 

-0.588 

-0.009 

SADARM 

0.120 

0.011 

0.686 

0.019 

0.266 

0.006 

PLS 

0.067 

0.011 

-0.058 

0.001 

-0.088 

-0.001 

T0W2B 

-0.030 

-0.001 

-0.093 

0.002 

0.193 

0.002 

VEMASID 

-0.063 

-0.001 

-0.056 

0.001 

0.053 

0.001 

VOL  Air 

0.207 

0.009 

-0.023 

0.000 

0.226 

0.012 

VOL  Gnd 

-0.106 

-0.002 

-0.004 

0.002 

-0.037 

0.000 

WAM 

0.496 

0.036 

0.235 

0.013 

0.076 

0.005 

155  ER 

-0.009 

0.003 

0.047 

0.000 

155  LtWt 

0.116 

0.006 

0.138 

0.003 

155  TGP 

0.176 

0.006 

0.233 

0.009 

ADVFIREFINDER 

0.109 

0.003 

-0.078 

-0.001 

AGS 

-0.325 

-0.005 

-0.377 

-0.003 

AH-64  LB 

0.353 

0.011 

0.418 

0.011 

AMS-H 

-0.116 

0.001 

-0.129 

-0.002 

ATACMS II 

0.122 

0.008 

-0.251 

-0.002 

GBS 

0.356 

0.009 

0.100 

-0.002 

LOSAT 

-0.193 

0.007 

0.316 

0.005 

MLRS  TGW 

0.662 

0.020 

0.631 

0.019 

NLOS-AT 

0.636 

0.016 

0.045 

0.003 

RAH-66 

1.563 

0.036 

0.835 

0.024 

STINGRAY 

-0.056 

0.003 

0.081 

0.005 

Adv  SEMA 

-0.136 

-0.002 

AFAS 

0.309 

0.012 

BLOCK  III 

0.185 

0.009 

CMV 

-0.287 

-0.005 

FARV-A 

0.090 

0.006 

FIFV 

0.685 

0.016 

RAH-66  LB 

0.998 

0.024 

^Already  fielded  to  all  units  in  this  scenario. 
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b.  The  following  summary  observations  are  made  on  the  Contingency  Scenario 
runs. 


(1)  Two  major  problems  were  found  in  Contingency  96  runs  and  corrected  for  all 
follow-on  scenarios.  The  wide  area  mine  system  (WAM)  was  being  used  three 
different  ways,  instead  of  two,  and  the  Bradley  chain  gun  was  not  being  modeled  to 
portray  kills  against  "lesser"  targets.  WAM  should  have  been  played  consistently 
"in”  or  "out”  of  the  generic  minefields;  but  when  the  VOLCANO  Multiple  Delivery 
Mine  System-Air  (VOL  Air)  and  WAM  were  both  present,  WAM  went  into  a  third 
minefield  belt.  This  caused  VOL  Air  to  receive  a  higher  coefficient  than  it  should  and 
the  Air  Volcano  has  been  manually  corrected  downward.  Another  problem  noted  was 
that  the  Bradley  chain  gun  used  only  the  antitank  (AT)  missile.  This  caused  prob¬ 
lems  for  the  TOW  2B  and  A.MS-H  as  the  failure  of  the  Bradley  to  kill  ATOM  ar  1 
"lesser"  targets  were  rolled  into  the  AT  missiles. 

(2)  A  minor  correction  between  1996  and  the  later  years  was  an  adjustment  to 
make  Blue  counterbattery  targeting  work  better. 

(3)  One  problem  discovered  after  all  the  runs  were  made  has  to  do  with  the 
improvement  of  the  Blue  base  cases  in  the  latter  years,  even  while  the  Red  modern¬ 
ized.  It  was  discovered  that  while  Red  upgraded  their  tanks  in  each  year,  the  capa¬ 
bilities  of  these  tanks  actually  decreased  due  to  a  data  input  error  with  the  older 
tanks.  This,  along  with  the  changes  described  above,  may  be  the  reason  why  the  base 
Blue  force  did  better  in  2004  and  2001  than  in  1996.  This  problem  was  corrected. 

(4)  The  number  of  Red  kills  by  Blue  seems  to  hit  a  threshold  (approximately 
5,000  kills)  and  cannot  get  much  higher.  The  reason  mt^y  be  that  Blue  consistently 
destroyed  all  Red  forces  forward  of  the  Blue  objective.  The  significant  change 
between  simulation  runs  is  the  number  of  Blue  causalities.  Usually,  the  highest 
number  of  Blue  casualties  is  in  the  base  case  run,  with  the  excursions  being  much 
lower.  This  causes  an  interesting  effect  on  the  FER  and  BFS.  Since  both  MOE  use 
Blue  causalities  as  an  intermediate  part  of  the  calculations  and  Blue  losses  is  the 
most  volatile  part  in  the  runs,  both  FER  and  BFS  are  highly  correlated.  Generally, 
this  means  that  the  rank-ordering  given  by  one  MOE  will  be  almost  the  same  as 
rank-ordering  done  by  the  other  MOE.  This  correlation  is  so  high  that  one  MOE  is  a 
linear  predictor  of  the  other  MOE. 

c.  The  following  summary  observations  are  made  on  the  Stressful  Scenario  runs. 

(1)  Red  smart  artillery  was  a  prominent  killer.  Likewise,  improvements  in 
Blue  counterbattery  assets  lessened  the  Red  effects. 

(2)  Blue  helicopters,  while  still  potent,  suffered  greatly  from  Red  air  defense 
assets. 

(3)  Improvements  in  close  combat  systems  became  much  more  important  in  this 
scenario.  Target  stealing  did  not  appear  to  be  as  much  of  a  problem,  as  this  was  a 
very  target-rich  environment. 

(4)  Blue  never  achieved  overwhelming  superiority  against  Red. 
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d.  The  following  individual  system  problems  were  discovered  in  all  scenarios,  and 
correction  experimental  designs  were  performed. 

(1)  Sense  and  destroy  armor  (SAD  ARM):  three  instead  of  six  submunitions, 
targeting  data  not  optimally  set,  and  orders  improperly  set  so  that  SAD  ARM  would 
not  fire  when  the  multiple  launch  rocket  system  terminally  guided  weapon  (MLRS 
TGW)  was  present  in  the  same  excursion. 

(2)  Nonline  of  sight  (NLOS):  improper  data  set  up  caused  it  to  fail  to  fire. 

(3)  Line  of  sight  antitank  (LOSAT):  improper  target  class. 

(4)  Armored  Gun  System  (AGS):  defaulted  to  an  improper  role  during  the 
counterattack. 

e.  Table  4-2  contains  the  Stressful  Scenario  case  results. 

f.  The  results  of  the  Stressful  Scenario  are  a  marked  contrast  to  the  Contingency 
Scenario.  The  Blue  forces,  in  the  Contingency  Scenario,  had  achieved  victory 
consistently.  In  the  Stressful  Scenario,  Blue  seldom  achieved  victory.  When 
comparing  the  experimental  design  runs  MOE  for  Contingency,  there  is  a  marked 
increase  of  MOE  values  above  the  base  case.  This  pattern  appears  linear  and  well- 
behaved.  Stressful  experimental  design  runs  do  not  have  linear  or  well-behaved 
patterns.  Both  Red  and  Blue  losses  vary  greatly,  as  do  the  MOE. 
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Table  4-2.  Stressful  Scenario  Results 
(contribution  of  systems  to  MOE) 

System  1996  2001  2008 


FER 

BFS 

FER 

BFS 

FER 

BFS 

INTERCEPT 

0.676 

0.563 

0.689 

0.486 

0.688 

0.454 

AAWS-M 

0.041 

0.016 

0.033 

-0.013 

0.041 

0.035 

ADATS 

-0.002 

0.001 

0.062 

0.032 

-0.058 

-0.017 

ADV  QUICKFIX 

0.015 

-0.019 

0.043 

-0.009 

-0.046 

-0.030 

ARV 

-0.014 

0.011 

0.016 

0.007 

-0.021 

0.001 

ATAS 

0.009 

-0.016 

-0.004 

-0.001 

-0.063 

-0.015 

AVENGER 

-0.030 

-0.016 

-0.036 

0.016 

0.051 

0.000 

FMTV  2.5/5 

-0.013 

0.011 

-0.002 

-0.023 

0.000 

-0.013 

GBCS  L/H 

-0.026 

-0.045 

0.007 

0.017 

0.001 

-0.006 

HAB 

0.022 

0.026 

0.034 

0.026 

0.036 

0.024 

MET 

-0.025 

0.009 

-0.006 

0.001 

-0.018 

-0.003 

JSTARS 

0.016 

-0.020 

0.002 

0.013 

0.038 

0.016 

M1A2 

0.041 

-0.004 

0.064 

0.012 

0.015 

0.032 

SADARM 

-0.030 

-0.027 

0.216 

0.025 

0.278 

0.056 

PLS 

0.000 

-0.011 

0.035 

-0.005 

0.067 

0.018 

TOW  2B 

0.009 

0.014 

-0.014 

0.006 

-0.026 

0.037 

VEMASID 

0.027 

0.004 

-0.006 

-0.023 

0.002 

0.009 

VOL  Air 

-0.010 

0.009 

-0.009 

-0.013 

0.052 

-0.012 

VOL  Gnd 

0.003 

0.006 

0.006 

-0.005 

-0.043 

-0.010 

WAM 

0.134 

-0.024 

0.047 

0.008 

0.040 

0.016 

155  ER 

0.027 

0.020 

0.028 

0.004 

155  LtWt 

-0.006 

-0.011 

0.034 

0.004 

155  TGP 

-0.016 

-0.019 

-0.036 

-0.024 

ADV  FIREFINDER 

0.043 

0.020 

-0.006 

0.004 

AGS* 

NA 

NA 

AH-64  LB 

0.093 

-0.006 

0.174 

0.033 

AMS-H 

0.029 

0.010 

0.019 

0.006 

ATACMS  n 

-0.013 

-0.008 

0.003 

0.013 

GBS 

0.047 

0.026 

0.098 

0.022 

LOSAT 

0.128 

0.023 

0.064 

0.012 

MLRSTGW 

0.026 

-0.011 

0.036 

-0.008 

NLOS-AT 

-0.048 

0.011 

-0.262 

-0.064 

RAH-66 

0.019 

-0.013 

0.157 

0.011 

STINGRAY 

0.011 

-0.010 

0.008 

0.007 

Adv  SEMA 

-0.008 

-0.004 

AFAS 

0.024 

0.001 

BLOCK  III 

0.218 

0.042 

CMV 

-0.098 

-0.012 

FARV-A 

0.024 

-0.006 

FIFV 

0.41C 

0.083 

RAH-66  LB 

0.212 

0.019 

*None  played  in  this  scenario  because  none  of  these  Blue  units  are  authorized  the 
Armored  Gun  System. 
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CHAPTERS 

IMPLICIT  MEASURES  OF  VALUE 


5-1.  GENERAL.  The  implicit  efTectiveness  portion  of  VAA  encompasses  .he 
identification,  gathering,  and  synthesis  of  information  necessary  to  provide  the 
appropriate  weighting  factors  for  use  in  calculating  predicted  marginal  values  for 
each  of  the  systems.  The  weighting  factors  will  be  used  in  a  multiattribute 
decisionmaking  environment  (TOPSIS)  in  order  to  generate  these  values.  The 
marginal  values  will  be  used  as  coefficients  of  an  objective  function  within  a  mi.xed 
integer  linear  programming  model  (VALOR). 

5-2.  METHODOl.OGICAL  OV'ERVTEW.  The  principal  notion  involved  in  the 
development  and  acquisition  of  these  weighting  factors  involves  the  use  of  the  AHP 
originated  by  Professor  Thomas  L.  Saaty  in  the  early  1970s.  In  this  approach, 
criteria  are  identified  at  one  or  several  levels  of  a  hierarchical  framework,  and  a 
pairwise  comparison  technique  is  used,  iteratively  entering  the  opinion  of  the 
surveyee  as  to  the  degree  one  criteria  is  more  important/  influential  than  the  other. 

In  doing  so,  the  surveyee  completes  a  positive,  reciprocal  matrix  of  these  opinions.  By 
calculating  the  eigen  or  characteristic  vector  of  the  matrix,  each  of  the  criterion  at 
the  current  level  of  the  hierarchy  is  associated  with  a  "proportion"  or  contribution  of 
the  criterion  to  the  next  higher  level  entry  of  the  hierarchy.  However,  unlike  other 
techniques  of  multicriteria  comparison,  the  AHP  method  provides  factors  which 
relate  to  each  other,  not  only  in  magnitude  (effectively  providing  an  ordinal  ranking), 
but  also  in  proportion  or  relative  magnitude  to  each  other.  It  is  this  characteristic  of 
proportionality  that  was  essential  for  use  in  the  later  optimization  phase  of  VAA. 

5-3.  AHP  HISTORY.  ProfessorThomasL.  Saaty  of  the  University  of  Pittsburgh  is 
most  noted  for  his  groundbreaking  DOD  and  State  Department  work  in  the  1970s 
where  his  AHP  principles  were  first  applied.  His  first  book  in  1980,  The  Analytic 
Hierarchy  Process,  was  well  received  and  was  followed  by  numerous  additional 
journal  publications:  Socio-Economic  Planning  Sciences  (12/86),  Mathematical 
Modeling  (08/87),  and  European  Journal  of  Operations  Research  (08/90). 

5-4.  CHARACTERISTICS  OF  AHP.  The  theory  behind  AHP  is  rich  in  substance, 
but  the  attributes  which  characterize  simply  the  elemental  qualities  of  the  method¬ 
ology  are  given  below: 

Useful  for  comparing  multiple  level/criteria  options 

Hierarchical  framework 

Characterized  by  pairwise  comparison 

Scale  of  input  values  between  l/9th  and  9 

Can  use  subjective  as  well  as  objective  data 

5-5.  AHP  ASSUMPTION.S  (REWORDED  AXIOMS).  The  assumptions  of  AHP 
provide  an  excellent  framework  for  an  introduction  to  the  technique.  It  is  assumed 
that  they  are  preceded  with  a  problem  involving  multiple  alternatives  to  be  compared 
and  a  defined  set  of  criteria  at  one  or  more  levels  with  which  to  base  the  comparison. 
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..  a.  No  alternative  has  infinite  importance.  If  an  alternative  had  infinite  value, 
then  it  is  not  an  alternative  but  rather  an  inviolate  condition  and  comparison  would 
not  be  necessary. 

b.  No  alternative  has  zero  importance.  Similar  to  subparagraph  a  above,  any 
alternative  having  zero  importance  is  not  an  alternative  and  is  immediately 
dominated  by  all  alternatives  which  have  any  importance  at  all. 

c.  Importance  ranges  from  equal  ( 1)  to  absolute  (9h  Although  controversial,  the 
graduations  of  the  preference  scale  consisting  of  the  integers  1  to  9  form  the  yardstick 
of  the  methodology.  A  number  of  related  efforts  have  shown  that  there  is  little  to  be 
gained  by  additional  graduations  and  that  the  value  of  1  associated  with  equality  is  a 
naturally  appealing  notion.  The  textual  descriptions  of  the  values  on  the  scale  are 
subject  to  revision  based  upon  the  context  of  the  criterion,  but  the  increasing 
value/importancc  as  the  numbers  increase  is  a  necessity. 

d.  Set  of  alternatives  is  not  subject  to  change.  Because  the  technique  involves  the 
combining  of  relative  importance  assessments,  the  removal  or  introduction  of  an 
alternative  can  have  a  significant  impact  on  the  final  results.  The  partitioning  of 
total  value  is  sensitive  to  the  number  of  parti tions-change  this  number  and  the 
process  is  apt  to  require  a  complete  reassessment. 

e.  Importance  is  reciprocal  between  any  two  alternatives.  The  relative  impor¬ 
tance  of  one  alternative  over  another  is  a  shared  ratio  relation.  If  A  is  more 
important  than  B  by  a  measure  of  5,  then  B  is  said  to  be  l/5th  as  important  as  A  by 
implication.  This  condition  allows  for  the  method  to  minimize  the  number  of 
comparisons  as  the  reverse  ordering  of  the  pair  of  alternatives  has  an  associated 
value  which  is  the  reciprocal  of  the  first.  Additionally,  an  alternative  has  equal 
importance  with  itself.  In  combination,  for  a  problem  with  n  alternatives,  these  two 
facts  generate  a  requirement  to  consider  only  (n  *  (n  - 1))  *  .5  pairwise  evaluations,  a 
significant  savings  over  n  *  n  comparisons. 

f.  All  criteria/alternatives  Impacting  on  a  problem  are  specifically  represented  or 
excluded.  This  is  the  most  difficult  aspect  of  the  construction  of  the  hierarchy.  It 
requires  that  at  any  level  in  the  hierarchy  where  there  are  competing  alternatives  or 
criteria,  the  items  comprise  a  set  of  possibilities  which  completely  covers  the  scope  of 
that  particular  level  of  the  hierarchy.  In  this  way,  this  assumption  relates  to  sub- 
paragraph  d  above  as  it  defines  the  number  of  partitions  occurring  at  a  given  level.  A 
more  subtle  implication  associated  with  this  assumption  is  one  which  requires 
maximum  independence  between  criteria^alternatives.  The  better  the  distinction 
between  alternatives,  the  easier  the  construction  of  the  hierarchy  becomes. 

5-6.  AHPTH  KORY  AND  MATHEMATICS.  With  the  completion  of  the  pairwise 
comparison,  the  real  work  begins  in  deriving  the  respective  weights  for  the 
criteria/alternatives  at  each  level  of  the  hierarchy.  The  evaluations  of  each  pairwise 
comparison  and  its  reciprocal  represent  components  of  a  system  of  relationships 
which  can  conveniently  be  portrayed  as  a  matrix  with  several  important 
characteristics.  The  first  characteristic  is  that  all  comparisons  of  an  alternative  with 
itself  equal  1  and  form  a  diagonal  of  I's.  Second,  all  elements  off  this  diagonal  are 
either  between  1  and  9  or  between  1/2  and  1/9  inclusive.  At  this  point,  there  exists 
what  is  called  a  positive  reciprocal  matrix  which  retains  a  number  of  useful 
characteristics  itself.  The  matrix  can  now  be  manipulated  to  extract  the  right 
eigenvector.  Also  called  the  characteristic  vector,  the  eigenvector  represents  an 
ordered  array  of  values  which  are  associated  with  the  system  of  relationships  found 
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within  the  matrix.  In  particular,  the  array  is  associated  with  one  characteristic  value 
(eigenvalue)  which  is  the  largest  of  all  n  eigenvalues.  The  calculations  associated 
with  the  derivation  of  the  eigenvector  are  generally  tedious  and  complex.  Saaty  often 
discusses  a  number  of  procedures  of  increasing  accuracy  of  estimating  these  values 
but  ascribes  to  the  most  accurate  method  of  that  of  the  following  expression: 


LIM 
A  » 


with  eT  =  (1,  1, ...,  1)  or  the  transpose  of  a  column  vector  of  I’s  and  A  represents  the 
positive,  reciprocal  matrix  described  earlier.  The  term  A^e  effectively  produces  a 
vector  of  row  sums  of  this  A^  matrix  while  the  remaining  portion  of  the  expression, 
(eTAke)-l,  generates  a  scaler  associated  with  the  sum  of  all  the  matrix  Ak  entries. 
While  in  theory,  the  values  of  the  resulting  vector  are  accurate  only  in  the  limit,  in 
practical  terms,  the  values  seem  to  stabilize  to  4  or  5  decimal  places  at  about  k  =  8. 

5-7,  CONSISTENCY.  In  addition  to  the  final  weighting  values  for  the  alternatives, 
the  AHP  methodology,  unlike  many  multiattribute  methods,  provides  a  measure  of 
inconsistency  in  the  form  of  a  consistency  ratio  (CR).  In  a  broader  sense,  it  is  obvious 
why  consistency  would  be  of  interest  in  general  for  an  application  of  multiple 
comparisons.  But  because  AHP  in  principle  is  sensitive  to  the  consistency  of  inputs,  a 
mathematical  derivation  of  a  measure  of  consistency  is  desirable. 

a.  General  Consistency.  A  simple  decomposition  of  consistency  would  include 
two  general  types  which  can  be  called  ordinal  and  technical  consistency.  Ordinal 
consistency  (also  called  transitivity)  is  that  condition  in  which  if  A  is  considered  more 
important  than  B  and  B  is  considered  more  important  than  C,  by  implication,  A  must 
be  more  important  than  C.  If  VAA  was  only  concerned  with  a  rank-ordering  of 
alternatives,  a  method  focusing  on  ordinal  consistency  would  be  sufficient.  However, 
relative  magnitude  or  proportion  is  critical  to  the  use  of  the  resulting  weights  in  the 
optimization,  and  so  technical  consistency  is  also  important.  Technical  consistency 
can  be  analogized  thusly;  if  A  is  3  times  as  important  as  B  and  B  is  2  times  as 
important  as  C,  A  must  be  6  times  as  important  as  C.  It  is  clear  that  during  the 
course  of  a  large  pairwise  comparison  survey,  perfect  consistency  of  both  types  would 
be  ideal.  It  is  also  clear  (both  from  Saaty's  and  the  study  team's  experience)  that  all 
surveyees  demonstrate  inconsistencies  to  some  degree.  The  ability  to  quantify  this 
consistency  or  lack  thereof  is  one  of  beneficial  facets  of  AHP;  the  consistency  ratio. 
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.  b.  Consistency  Ratio.  The  consistency  ratio  is  a  ratio  of  an  input  matrix 
dependent  index  called  the  consistency  index  (Cl)  and  an  index  of  a  notional  average 
matrix.  Both  indices  are  also  ratios  which  are  based  on  the  assumption  that  a 
perfectly  consistent  input  matrix  has  a  max  eigenvalue  equal  to  the  number  of 
alternatives.  When  the  difference  is  between  the  max  eigenvalue  and  the  number  of 
alternatives  is  standardized  by  dividing  by  the  degrees  of  freedom  (n  -  1),  the 
resulting  value  is  theorized  to  represent  an  index  of  the  measurable  consistency.  In 
mathematical  terms, 


CZ  =  (\  —  n)/(n  —  1), 

niax 


with  Amax  being  the  max  eigenvalue.  The  max  eigenvalue  is  calculated  for  the  input 
matrix  of  the  responses  to  the  survey.  Once  obtained,  the  Cl  is  divided  by  an  average 
Cl  of  a  large  quantity  of  input  matrices  of  the  same  order  as  the  current  problem 
which  were  constructed  with  random  entries.  The  ultimate  effect  of  this  division  is 
the  generation  of  a  ratio  which  reflects  how  closely  the  input  corresponds  with  a 
survey  completed  with  totally  random  responses.  As  the  ideal  index  is  0  and 
increases  with  deviations  from  the  ideal,  the  consistency  ratio  is  probably  more  aptly 
named  the  inconsistency  ratio. 

c.  Acceptable  Inconsistency.  According  to  Saaty,  a  convention  of  rejecting 
surveys  having  a  CR  in  excess  of  0.1  has  met  with  general  acceptance.  In  the  VAA 
experience,  the  aggregated  survey  inputs  proved  to  be  significantly  free  of 
’nconsistencies,  generally  having  CRs  on  the  order  of  0.03.  This  is  attributable  to  a 
serendipitous  factor  of  using  a  geometric  mean  technique  of  survey  aggregation 
(which  deserves  further  study).  It  should  be  noted  that  the  indi v  idual  surveys  were 
not  as  consistent,  and  techniques  for  assisting  the  survey  taker  in  maintaining 
consistency  would  be  of  benefit.  In  summary,  the  excellent  CRs  provided  an 
additional  sense  of  credibility  to  the  final  weights. 

5-8.  HIE  RA  RCH  Y.  The  hierarchy  constructed  for  VAA  11  was  fundamentally 
different,  both  from  the  standard  AHP  as  well  as  that  used  in  VAA  I,  in  response  to 
the  sponsor; user  requested  desire  to  see  how  the  systems  fared  in  various  perspectives 
of  performance  or  benefit.  Saaty's  hierarchy  was  characterized  by  multiplicative 
branching  as  one  progressed  out  from  the  root,  topmost,  or  level  1  (goal)  of  the 
hierarchy,  each  node  having  a  fixed  number  of  contributing  elements.  The  design  of 
the  hierarchy  in  VAA  I  also  mirrored  this  structure.  In  contrast,  the  hierarchical 
tree  shown  in  Figure  5- 1  allows  for  evaluation  of  different  values  for  given  systems  if. 
for  example,  interest  was  focused  on  only  a  selected  three  elements  in  the  third  or 
fourth  level  (VAA  coefficient  =  level  1)  at  the  exclusion  of  the  others.  It  should  be 
pointed  out  that  the  surveys  and  AHP  application  stop  short  of  the  systems  level 
where  a  multicriteria  decisionmaking  tool,  TOPSIS,  is  used.  This  tool  and  its  use  are 
described  in  C..apter  6. 
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VAA  COEFFICIENT 


Figure  5-1.  Hierarchical  Tree 


a.  Implicit  versus  Explicit.  The  second  level  of  the  hierarchy  consists  of  the 
division  of  system  value  into  two  effectiveness  components,  explicit  and  implicit. 
Explicit  measures  of  effectiveness  can  be  defined  as  factors  that  measure  the  worth  of 
the  system/program  in  terms  of  its  contribution  to  overall  force  effectiveness  and  that 
can  be  directly  and  objectively  computed  (e.g.,  combat  simulation  results).  Implicit 
measures  of  effectiveness  are  factors  that  affect  the  decisionmaking  process,  but  are 
only  subjectively  measurable  (e.g.,  political  risk). 

b.  Timeframe  and  Scenario.  VAA  II  is  targeted  toward  the  1994-99  POM.  It  is 
consistent  with  this  purpose  to  consider  systems  and  programs  which  will  potentially 
begin  or  expand  their  research,  development,  test,  and  evaluation  (RDTE)  outlays 
within  this  5-year  window.  Additionally,  the  EPP  was  a  consideration,  providing  an 
additional  8  years  of  interest.  This  prompted  the  need  for  a  13  +  year  look  which  was 
decomposed  into  three  periods:  near-  (1996-2000),  mid-  (2001-2007),  and  far-term 
(2008  and  beyond).  Taking  the  first  year  of  each  period  as  representative,  the 
chronological  element  of  system  value  added  was  derived.  Secondly,  as  the  Army 
continues  to  be  a  worldwide  deployable  organization,  a  representative  cross-section  of 
tactical  environments  was  desired.  This,  too,  was  divided  into  three  areas:  SWA, 
NEA,  and  Central  Europe  (post-CFE)  which  generally  depicted  the  primary  mid-  and 
high-intensity  combat  scenarios  under  analysis  in  1991.  It  was  felt  that  each 
scenario  represented  sufficiently  different  enemy,  environment,  command,  control, 
communications,  computers,  and  intelligence  (C 41),  and  deployability  aspects  to 
justify  their  inclusion  while  consciously  minimizing  the  number  of  scenarios. 
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(1)  In  combination,  the  timeframes  and  scenarios  composed  an  array  of  nine 
distinct  time  and  location  pairings  which  was  believed  would  provide  a  suitable  and 
comprehensive  portrayal  of  the  contributions  of  the  systems.  Additionally,  it  was  felt 
that  both  the  survey  and  the  combat  modeling  would  be  capable  of  addressing  this 
limited  number  of  timeframe  and  scenario  combinations. 

(2)  In  the  survey,  the  surveyees  were  asked  to  rate  the  relative  importance  of 
each  pair  of  scenario-year  combinations  from  the  perspective  of  the  need  for 
programs'systems  for  inclusion  in  the  1994-99  POM  development. 

c.  Secondary  P'actors.  The  six  factors  which  comprised  the  qualitative  or 
implicit  portion  of  the  hierarchy  are  defined  below  as  they  appeared  in  the  survey 
briefing.  The  hypothetical  examples  which  follow  the  factor  definitions  were  used  to 
exemplify  some  aspects  of  the  notions  associated  with  the  factors  and  represent  only 
analogies  or  situations  having  the  characteristics  previously  defined. 

(1)  Political  Risk.  The  subjective  evaluation  of  the  supportive  or  non- 
supportive  attitude  prevalent  across  the  scope  of  public.  Congressional,  executive, 
and  Army  interest  groups  regarding  the  proposed  program.  Greater  political  risk  is, 
by  convention,  less  desirable. 

HYPOTHETICAL  EXAMPLES.  Recent  environmental  accidents  could 
generate  public  sentiment  unfavorable  to  diesel-based  ground  vehicular  systems. 

The  Senate  Armed  Services  Committee  announces  its  fundamental  distrust  of 
millimeter  wave  technology.  Major  command  commanders  voice  a  unified  desire  to 
reduce  types  of  vehicles. 

(2)  Programmatics,  A  subjective  measure  of  the  aspects  of  the  proposed 
program  which  would  influence  program  execution.  Issues  of  complexity,  flexibility, 
and  executability  of  the  program  are  considered.  By  convention,  greater  complexity 
of  programmatics  is  less  desirable. 

HYPOTHETICAL  EXAMPLES.  A  production  cycle  which  includes 
maintaining  a  warm  production  line  for  2  years  midway  through  the  program  would 
be  of  increased  complexity.  The  cost  of  increasing  or  decreasing  production  in  years 
3-5  are  overwhelmingly  unfavorable,  hence  indicating  inflexibility. 

(3)  Asset  Versatility  and  Deployability.  A  measure  of  the  applicability  of  a 
program  to  multiple  theaters  of  operation  and  their  associated  environments  and 
requirements. 

HYPOTHETICAL  EXAMPLES.  A  program  to  develop  a  small  unit  snow 
vehicle  would  likely  have  limited  versatility.  Light  vehicle-packed  demineralization 
and  desalinization  equipment  would  likely  have  widespread  versatility  and 
deployability. 
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(4)  Operational  and  Technological  Risk.  A  subjective  measure  of  the 
probability  associated  with  a  program  meeting  all  of  its  stated  performance  criteria. 

HYPOTHETICAL  EXAMPLES.  A  program  relying  on  superconductor 
technology  portends  considerable  technological  risk.  A  program  which  will  only  work 
if  all  of  its  5,000  individual  mechanical  components  have  a  minimum  mean  time 
between  feilures  of  300  hours  would  likely  be  conr'dered  of  high  operational  risk. 

(5)  Criticality  of  Need  as  Related  to  Current  Capability.  Program  need  is  a 
subjective  measure  of  the  current  status  of  the  mission  area  or  function  associated 
with  a  program  and  the  relative  fraction  cf  improvement  that  is  expected. 

HYPOTHETICAL  EXAMPLES.  A  tank  round  which  penetrates  an 
additional  5mm  of  armor  is  not  as  critical  as  compared  to  nuclear,  biological,  and 
chemical  (NBC)  suit  which  could  be  recycled  after  wear  with  i.o  loss  of  effectiveness  if 
not  exposed  to  contaminants. 

(6)  Impact  on  Sustainability  (combat).  A  subjective  measure  of  the 
collective  effect  of  the  system  on  other  systems  which  support  the  effort  of  the  force. 

By  convention,  a  view  toward  lessening  the  sustainability  burden  and  in  tui  n 
enhancing  the  sustainability  o^  “'he  force  will  be  used  as  being  more  desirable. 

HYPOTHEl’lCAL  EXAMPLES.  A  vehicle  system  that  is  self-recwering  or 
that  carries  substantially  more  of  its  primary  munitions,  effectively  negating  the 
need  for  support  assets,  would  likely  have  a  desirable  impact,  whereas  a  system 
requiring  a  new  family  of  support  vehicles  and  equipment  would  likely  have  a  less 
desirable  impact. 

d.  BPS  versus  PER  Measures  of  Effectiveness.  In  this  survey,  which  required 
only  one  input,  a  graduation  of  the  "insightfulness"  or  utility  of  the  two  measures  of 
effectiveness  which  could  be  explicitly  measured  was  obtained.  These  two  measures 
of  effectiveness  were  readily  available  from  the  CORBAN  simulation  runs.  The 
definitions  of  the  two  MOE  were  given. 

(1)  Blue  force  survivability  (BPS)  -  a  value  of  effective  Blue  force  combat 
capability  remaining  Upon  termination  of  the  simulated  conflict. 

(2)  Fractional  exchange  ratio  (FER)  -  the  ratio  of  the  proportion  of  Red  systc.ms 
lost  to  the  proportion  of  Blue  systems  lost. 

It  was  then  noted  that  Blue  force  survivability  is  essentially  a  measure  of  simple 
survivability  while  fractional  exchange  ratio  imparts  a  measure  of  the  lethality  of 
the  Blue  force  with  respect  to  the  ratio  of  Blue  to  Red. 

e.  Feedback  from  Surveyees 

(1)  An  aspect  of  the  survey  process  which  grew  in  significance  as  the  surveys 
were  conducted  was  the  feedback  or  narrative  comments  provided  by  the  parti¬ 
cipants.  Originally,  the  study  team  was  primarily  concerned  with  technical  aspects 
of  the  survey  process  and  the  VAA  methodology.  It  was  desired  to  obtain  opinions  of 
items  specifically  associated  with  the  survey  such  as  the  mechanics  of  how  the  survey 
was  conducted,  the  nature  of  the  survey  components,  the  definitions  or  vernacular 
used,  suggestions  fo’'  improvement,  and.  finally,  the  degree  of  confidence 
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in.the  comprehensive  VAA  program.  It  became  increasingly  obvious  that  the 
feedback  exceeded  the  above  dimensions. 

(2)  In  order  to  promote  the  unrestrained  evaluation  of  survey  elements,  each 
surveyee  was  informed  of  the  nonattribution,  close  hold  handling  of  the  survey  inputs 
(see  Appendix  D,  VAA  Read-ahead  Memo).  In  this  vein,  under  any  circumstance  will 
the  identification  of  a  surveyee,  in  association  with  his/her  respective,  completed 
survey,  be  disclosed.  Efforts  were  made  to  strip  identification  from  the  data  during 
the  processing,  and  it  is  often  impossible  to  reconstruct  the  associations.  It  is  believed 
that  this  condition  of  the  survey  was  the  most  constructive  and  brought  out 
significantly  more  insight  than  the  surveys  themselves. 

5-9.  SURVEYS.  Two  surveys  are  used  within  the  Value  Added  Analysis 
framework:  decisionmaker  surveys  and  subject  matter  expert  (SME)  surveys. 

a.  Decisionmaker  Surveys 
(1)  Construction 

(a)  Survey  Mechanisms.  The  central  element  for  the  execution  of  the 
survey  is  the  pairwise  comparison.  In  addition  to  a  decision  of  which  alternative  is 
most  important,  the  AHP  requires  a  value  judgment  of  the  degree  of  greater 
importance.  It  was  intrinsically  important  to  include  mechanisms  which  addressed 
both  of  these  issues.  A  paper  version  of  the  survey  was  constructed,  a  sample  of  which 
is  shown  in  Figure  5-2. 


PROGRAMMATICS 

A  subjective  measure  of  the 
aspects  of  the  proposed  program 
which  could  influence  program 
execution.  Issues  of  complexity, 
flexibility,  and  executability  of 
the  program  are  considered.  By 
convention,  greater  complexity 
of  programmatics  is  less 
desirable. 


CRITICALITY  OF  NEED 
AS  RELATED  TO 
CURRENT  CAPABLITY 

Program  need  is  a  subjective 
measure  of  the  current  status  of 
the  mission  area  or  function 
associated  with  a  program  and 
the  relative  fraction  of 
improvement  that  is  expected. 


VS. 


Absolute  Very  Strong  Weak  Equal  Weak  Strong  Very  Absolute 

Strong  Strong 

□  □□□□□□□□□□□□□□□□ 

Check  the  box  which  you  believe  is  presently  the  best  measure  of  the 
importance  of  the  item  on  the  left  as  compared  to  the  item  on  the  right . 


Figure  5-2.  Survey  Sample 
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This  appeared  to  be  an  adequate  construction.  The  survey  instrument  displayed  the 
two  alternatives  simultaneously.  It  provided  a  textual  description  or  definition  of  the 
alternatives,  identifying  any  necessary  conventions.  The  mirrored  scale  at  the 
bottom  gave  a  spatial  effect  which  provoked  the  desired  sequential  method  of 
response  completion;  determining  first  if  the  left  or  right  item  was  more  important, 
using  the  associated  side  of  the  scale  to  record  to  what  degree  the  selected  item  was 
more  important. 

(b)  Th  3  survey  instruments  were  printed  on  individual  5x7  cards  and 
consolidated  into  ordered  packets,  held  together  by  rings.  The  assembled  survey 
provided  historical  evidence  to  the  exact  input  of  each  member  of  the  surveyed 
population.  The  instruments  were  identified  by  alphanumeric  coding  which 
precluded  identification  of  surveyees  in  conjunction  with  their  answers. 

(2)  Execution 

(a)  Identification  of  Surveyees.  An  operational,  as  well  as  a  technical 
evaluation  and  perspective,  was  desired  of  those  surveyed.  Immediate  efforts  were 
made  to  locate  both  key  military  and  civilian  individuals  having  an  integral 
association  with  the  planning,  selection,  or  utilization  of  the  systems  in  contention 
for  POM  dollars.  This  ultimately  resulted  in  a  group  of  significant  size  spread 
throughout  the  Army. 

(b)  Timeframe.  Decisionmaker  surveys  were  conducted  during  the  period  22 
August  to  8  November  1991  and  consisted  of  scheduled  30-minute  blocks  of  time  for  a 
short  VAA  overview  briefing  followed  by  the  surveys.  The  surveys  were  conducted  in 
isolated  settings  with  only  the  surveyee  and  survey  team  members  present.  The  list 
of  surveyees,  shown  in  Table  5  1,  was  later  extended  to  include  major  command 
deputy  chiefs  of  staff  for  resource  management  (DCSRMs),  selected  general  ofilcers, 
and  senior  DA  civilians.  (See  Appendix  D,  Memorandum,  Identification  of  VAA 
Decisionmaker  Surveyees,  1  Aug  92). 


Table  5-1. 

Surveyees 

ASA(RDA) 

DUSA(OR) 

Deputy  ASA  (Plans  and  Programs) 

Military  Deputy  to  the  ASA(RDA) 

Comptroller  of  the  Army 

DCSOPS 

DCSLOG 

DISC4 

DAS 

OCSPER 

OCSINT 

Director,  PAED 

ADCSOPS  (Force  Development) 

(3)  Geometric  Mean  Pooling.  The  AHP  is  designed  to  provide  a  single 
weighting  value  for  all  alternatives  at  every  level.  Normally,  this  is  done  on  a  single 
occasion,  and  if  involving  a  group  of  individuals,  it  is  done  in  a  group  setting  with 
debate  and  discussion  on  each  pairwise  assessment.  Because  the  Value  Added 
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Analysis  was  targeted  toward  the  HQDA  level,  it  was  immediately  apparent  that  it 
would  be  impossible  to  have  the  entire  survey  population  together  for  such  a  process. 
This  led  to  an  aggregating  approach  which  used  the  geometric  mean  of  all  responses 
for  a  given  pairwise  comparison  as  a  component  of  a  composite  survey.  The  geometric 
mean  is  the  nth  root  of  the  product  of  the  responses.  The  geometric  mean  seemed  to 
be  intuitively  appealing  as  it  tended  to  coincide  with  the  "center  of  mass"  opinion  of 
the  ^oup  and  would  be  less  affected  by  extreme  or  outlying  responses  as  the  arith¬ 
metic  mean  would  be.  As  for  not  being  done  in  a  group  environment,  it  was  concluded 
that  this  separate  but  pooled  technique  actually  enhanced  the  survey’s  validity  as 
group  and  interpersonal  dynamics  were  precluded. 


(4)  Results.  The  following  are  the  results  from  the  surveys  given  for  VAA 
Phase  n. 

Effectiveness  weights: 


Implicit  0.2303 

Explicit  0.7696 

Implicit  criteria  weights: 

Political  risk  0.0578 

Programmatics  0.0655 

Asset  versatility  and  deployability  0.2380 

Operational  and  technical  risk  0.1734 

Criticality  of  need  0.3 135 

Impact  on  sustainability  WRT  combat  0.1516 

Explicit  criteria  MOE  weights: 

Fractional  exchange  ratio  0.4653 

Blue  force  surviving  0.5346 


(5)  Sample  Feedback  from  Surveyees.  Feedback  is  provided  to  demonstrate 
the  depth  of  personal  opinion  which  was  provided  by  the  survey  participants. 

•  Justifying  the  Armed  Forces  is  more  pressing  a  need  than  justifying  the 
Army  materiel  requirement  of  the  future. 

•  With  respect  to  timeframes  and  scenarios,  I  would  have  a  much  different 
weighting  if  the  survey  was  given  in  the  context  of  potential  for  occurrence  of  combat. 

•  Those  surveyees  not  having  DCSOPS  or  PAE  experience  should  not  have 
their  answers  count  as  much. 

•  I  hate  political  elements  in  thes'’  (budgeting)  processes  and  I  ignore  them. 
(Note:  political  risk  was  not  the  5th  or  6th  ranked  secondary  factor  for  this 
individual.) 

b.  Consolidated  Feedback.  Because  of  the  variety  and  breadth  of  the  feedback 
as  well  as  similar  comments  repeated  among  several  individuals,  a  compendium  of 
feedback  entries  is  provided  below. 
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(DA  number  of  surveyees  provided  opinions  on  the  type  of  presentation  that 
should  be  used  to  display  the  results  of  Value  Added  Analysis.  This  included  showing 
a  breakdown  between  program,  RDTE,  and  sustainment  costs  with  the  program 
items  displaying  priority,  quantity,  and  position  on  the  program  timeline.  It 
appeared  that  money  distribution  by  year  and  the  related  breakdown  of  system  costs 
and  force  benefits  was  desired.  This  would  be  one  of  the  most  difficult  displays  of 
information  to  compose  successfully. 

(2)  Data  was  commonly  viewed  as  the  cornerstone  of  the  VAA  and  POM 
process.  As  such,  it  was  suggested  that  the  Secretary  of  the  Army  (Research, 
Development,  and  Acquisition)  (SA(RDA))  be  principally  involved  in  providing 
salient  and  timely  baseline  program  data  such  as  minimum  sustaining  rates,  current 
production  rates,  maximum  rates,  economic  order  quantities,  and  minimum 
programs  required.  This  type  of  information  would  also  allow  some  additional 
insight  into  program  flexibility. 

(3)  VAA  should  establish  a  correlation  of  costs  with  the  secondary  factors. 

(4)  Some  surveyees  were  concerned  with  fleet  age  (in  conjunction  with  the 
optimization),  especially  RAM  (reliability,  availability,  and  maintainability)  and 
associated  technological  implications. 

(5)  It  was  reiterated  on  several  occasions  that  VAA  was  working  on  "big  dollar 
items/programs,"  so  the  methodology  must  be  credible. 

(6)  Some  surveyees  questioned  the  specific  things  (in  order  of  impact)  that 
affect  RDA.  In  particular,  does  the  Ofilce  of  the  Secretary  of  Defense  (OSD)  (an 
external  influence)  or  elasticity  of  program  have  the  greatest  effect? 

(7)  Most  of  the  fiscal  orientation  of  VAA  is  on  what  is  the  best  program  to  get 
for  a  prescribed  TOA.  A  look  at  the  marginal  value  of  additional  or  less  TOA  would 
open  an  entirely  new  avenue  of  analysis. 

(8)  Force  structure  was  identified  as  being  an  important  variable  in  Value 
Added,  most  often  fixed  by  assumption.  Can  VAA  look  into  the  sensitivities  of 
varying  force  structure  as  well  as  program?  And  how  are  reserves  considered? 

(9)  Current  POM  development  is  centered  on  mission  and  functional  areas, 
each  with  their  own  proponents  and  offering  area  fixes.  VAA  should  attempt  to 
transcend  this,  not  only  with  a  macro  view  but  a  magna-view  (incorporate 
nonwarfighting  considerations)  of  how  to  do  this  (program  building)  better. 

(10)  Several  surveyees  spoke  of  how  they  felt  that  SWA  has  not  yet  settled 
down,  despite  growing  attitudes  to  this  effect.  They  said  that  SWA  may  be  at  the  top 
of  many  people’s  list  in  the  present  and  falls  to  last  in  the  mid-  and  far-terms,  but 
they  believe  this  to  be  erroneous.  One  facet  of  concern  is  that  in  SWA,  the  enemy  is 
not  being  looked  at  in  the  right  light;  the  opposition  will  be  much  more  modernized 
and  will  often  look  like  the  US  equipmentwise. 

(1 1)  One  individual  remarked  that  the  issue  of  sustainability  used  to  irritate 
him  when  he  was  an  officer  at  the  tactical  level,  but  now  it  has  overwhelming 
importance  in  a  strategic  sense.  He  was  happy  to  see  it  included  within  the 
framework  of  VAA. 


7- 


CAA-SR-92-10 


(12)  A  recurring  theme  of  fear  of  "hollow"  forces  was  found  in  the  survey 
population.  The  term  "hollow"  appeared  to  hold  different  implications  to  different 
individuals. 

(13)  One  particularly  interesting  discussion  centered  on  the  existence  of  a 
gestation  period  for  systems.  This  period  is  one  in  which  money  needs  to  be  spent 
despite  the  fact  that  the  potential  and  integration  issues  of  the  system  are  not  yet 
realized. 

(14)  With  the  notion  of  leap  ahead  technological  goals  in  vogue,  one  individual  • 
qualified  the  current  most  pressing  problem  as  one  of  deciding  do  we  live  with  what 
we  have  in  the  mid-term  so  that  we  can  afford  greater  payoff  items  in  the  far-term,  or 
do  we  buy  for  both  mid-  and  far-term  but  on  a  smaller  scale? 

(15)  Several  individuals  thought  that  certain  personal  characteristics  were 
necessary  in  the  surveyees  in  order  to  provide  worthwhile  information  during  the 
conduct  of  the  surveys.  There  was  a  concern  that  other  surveyees  may  not  have  the 
proper  view  or  perspective  to  provide  meaningful  input  in  the  POM  building  process. 
In  the  words  of  one  surveyee,  "You  don't  know  what  you  don't  know." 

(16)  Some  held  that  NEA  was  of  great  importance  and  would  continue  to  rise  to 
most  important  over  time.  Especially  in  economic  terms,  the  importance  of  the 
Pacific  basin  will  change  operating  requirements  for  systems.  Much  of  the  current 
systems  are  not  optimized  for  fighting  in  Asia. 

(17)  It  was  pointed  out  in  one  discussion  that  the  warfight  in  SWA  is  the  same 
as  that  of  CFE,  the  most  prominent  difference  being  the  element  of  navigation  where 
SWA  was  more  difficult.  It  was  concluded,  however,  that  CFE  proves  to  be  a  more 
stressful  environment.  This  added  stress  was  empirically  evidenced  by  acceptable 
effective  exchange  rates  on  the  order  of  4  or  6  to  1  in  Europe  but  now  400-1,000  to  1  in 
SWA. 

(18)  The  biggest  dilemma-how  to  articulate  need  for  military  in  light  of 
public/Congressional  opinion  of  a  world  free  of  threats  and  defining  the  forces  needed 
to  accomplish  this  new  mission. 

(19)  This  (the  POM  building  process)  is  a  political  decision  in  the  long  run. 

(20)  Modernization  is  more  important  as  the  force  gets  smaller. 

(21)  1974  was  a  watershed  year  in  the  way  things  were  looked  at  in  the  Army. 
The  seeds  of  the  Gulf  War  were  sown  in  this  timeframe  because  a  more  holistic  look 
was  taken.  What  the  Army  must  have  now  is  a  larger  perspective--a  world  view 
(useful  everywhere). 

(22)  Suggestions  were  offered  regarding  measures  of  effectiveness:  quickness, 
decisiveness,  and  minimum  casualties. 
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(23)  Lethality  and  mobility  have  increasing  importance  as  a  function  of  time 
and  should  have  increasing  weight  accordingly. 

(24)  There  is  a  corps-level  orientation  (not  division). 

(25)  Capital  investment  strategy  should  be  a  leap  ahead  if  there  is  not  a 
compelling  threat. 

(26)  Requirements  should  be  first  (referring  to  the  requirement-based 
acquisition  priority). 

(27)  The  US  may  be  faced  with  a  Third  World  power  who  has  bought  British, 
German,  or  other  Western  equipment.  Therefore,  very  high  tech  armies  may  be 
faced. 

(28)  Stability  of  RDTE  and  RDA  as  a  whole  are  especially  important. 

(29)  Look  at  operating  and  support  (O&S)  costs  when  procurement 
commitments  are  made.  Tradeoff  between  savings  achieved  by  new  systems  not 
obtained  because  old  systems  had  to  be  kept.  Need  to  trade  off  between  force 
structure  and  equipment.  Eliminate  high-low  mixes  of  modernized  systems.  All 
units  should  be  equipped  the  same. 

(30)  Political  risk  could  be  overriding.  Could  this  be  a  supervalue  criterion? 

(31)  At  least  one  surveyee  thought  the  higher  the  rank  of  the  survey 
population,  the  better  the  quality  of  the  answers,  but  he  agreed  that  the  anonymity  of 
the  survey  process  lent  itself  well  to  increasing  the  value  of  any  of  the  surveys. 

(32)  The  six  secondary  factors  equate  to  Mr.  Convers’  four  characteristics  of  a 
viable  system;  (1)  validated  need,  (2)  approvable  (in  Pentagon  and  Congress 
perspective),  (3)  program  executable,  and  (4)  affordable  (cost). 

(33)  In  discussing  alternatives  to  LER/FER;  how  fast  are  objectives  achieved, 
total  combat  time,  and  total  force  applied  defined  as  area  versus  time? 

(34)  One  significant  problem  concerns  the  long  time  used  to  develop  systems. 
Even  though  the  US  thinks  near-term,  there  is  no  long-range  development  thinking. 

(35)  There  is  no  need  for  a  2008  timeframe  in  VAA  as  there  are  too  many  things 
which  are  undefined  and  extrapolated  (e.g.,  threat). 

(36)  In  terms  of  scenario,  a  concurrent  contingency  scenario  is  needed  (any  one 
contingency  scenario  does  not  stress  our  capabilities). 

c.  Decisionmaker  Survey  Assessment.  Across  the  board,  these  participants 
were  knowledgeable  and  demonstratively  capable  individuals.  If  the  quality  of  the 
results  is  characterized  by  the  quality  of  the  participation  of  the  survey  population 
and  the  veritable  wealth  of  insight  proffered,  then  it  can  be  said  that  the  quality  was 
superb. 
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5:J0.  SUBJECT  MATTER  EXPERT  (SME)  SURVEY 

a.  Development.  The  SME  survey  was  designed  to  provide  the  assessments 
(values)  of  the  alternatives  in  terms  of  their  respective  success  in  fulfilling  the  most 
favorable  aspects  of  the  six  secondary  factors  previously  defined.  Originally,  this  was 
envisioned  to  be  a  pairwise  comparison  activity  similar  to  the  decisionmaker  survey 
but,  owing  to  the  number  of  alternatives  and  the  resulting  quantity  of  questions,  it 
was  felt  that  SME  effort  would  be  overwhelming  and  unrealistic.  The  survey  finally 
completed  consisted  of  a  scoring  of  each  of  the  systems  in  terms  of  the  secondary 
factors. 

b.  Execution.  A  survey  package  was  compiled  which  included  an  instructional 
memorandum,  read-ahead  memo,  long  and  short  system  title  and  definitions,  and 
survey.  The  system  title  and  definitions,  in  addition  to  serving  as  a  convention  for 
nomenclature,  were  used  as  an  input  device  to  allow  the  surveyees  to  provide 
additional  comments  on  the  actual  or  current  composition  of  the  program. 

(1)  The  survey  population  was  identified  as  being  the  various  offices  of  the 
DCSOPS.  Each  of  the  individual  ofTices  was  tasked  with  the  responsibility  of  stafTing 
and  completing  respective  elements  (programs  applicable)  of  their  surveys.  This 
staffing  was  to  include  appropriate  communications  with  project  managers,  system 
managers,  applicable  organizations  of  the  Secretariat,  and  other  Headquarters, 
Department  of  the  Army  level  organizations  having  knowledge  of  the  programs.  The 
ultimate  goal  was  to  provide  a  multilateral  assessment  at  the  "expert"  level  of  the 
current  state  of  the  program  within  the  contexts  of  the  six  secondary  factors. 

(2)  A  example  portion  of  the  SME  survey  package  is  displayed  in  Figure  5-3. 


SUBJECT  MATTER  EXPERT  SURVEY  INSTRUCTIONS 

Consider  each  of  the  following  systems  under  the  secondary  factor  given.  Determine  to  what  extent  the 
system  embodies  the  most  positive  aspects  of  the  factor.  Indicate  your  evaluation  by  making  an  X  in  the 
appropriate  square  on  the  scale  to  the  right  of  the  system  name.  Note  that  the  scale  value  increases  (to  the 
right)  with  desirability. 

SECONDARY  FACTOR:  PoliUcal  Risk 

The  subjective  evaluation  of  the  supportive  or  non-supportive  attitude  prevalent  across  the  scope 
of  public,  congressional,  executive,  and  Army  interest  groups  regarding  the  proposed  program. 
Greater  political  risk  is,  by  convention,  less  desirable. 
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Figure  5-3.  Example  -  Survey  Instructions 
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c.  Results.  The  final  scores  for  the  systems  as  they  relate  in  each  of  the 
secondary  factors  are  given  in  Table  5-2.  Note  that  the  convention  is  the  larger  the 
number,  the  more  favorable  the  evaluation. 

Table  5-2.  Secondary  Factors  Survey  Results 
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5:11.  FUTURE  DEVELOPMENTS.  A  number  of  lessons  were  learned  during  the 
evolution  of  the  surveys  which  lend  themselves  to  changes  to  the  process  in  order  to 
improve  it. 

a.  The  mechanics  of  the  survey  process  revealed  an  automated  survey  could  be 
envisioned  as  substantially  more  beneficial  as  compared  to  the  current  paper  process. 
The  primary  benefits  would  include  sequential  breakdown  of  the  pairwise 
comparison  elements,  error  checking  during  e.xecution,  feedback  of  survey  results 
upon  completion,  and  less  human  intervention  in  the  manipulation  of  the  final 
survey  data.  One  possibility  is  the  development  of  an  artificial  intelligence/expert 
system  module  designed  to  assist  the  surveyee  in  the  maintenance  of  consistency  or 
recording  the  specific  rationale  for  the  inconsistencies. 

b.  The  subject  matter  expert  survey  design  was  a  departure  from  the  pairwise 
comparison  approach  because  the  number  of  comparisons  would  be  significant,  and 
the  scope  of  subject  matter  expertise  is  often  very  confined.  A  procedure  in  which  a 
large  number  of  SME  participants  could  provide  independent  and  narrow  scope 
assessments  has  been  devised.  Currently  referred  to  as  the  shingles  approach,  such  a 
mechanism  would  access  a  much  larger  space  of  expertise  with  a  reasonable  quantity 
of  comparisons. 

c.  One  particular  strategy  of  analysis  which  was  not  conducted  was  the 
breakdown  of  the  assessments  by  aggregating  inputs  categorically.  Some  categories 
of  possible  interest  would  be  branch  affiliation,  flag  versus  field  grade  versus  civilian, 
category  of  current  position  (e.g.,  logistic),  major  Army  command  (MACOM)  versus 
ARSTAF,  and  recent  (post-Vietnam)  combat  experience  versus  no  combat  experience. 

d.  A  final  possible  development  is  an  assessment  mechanism  of  individual,  often 
subconscious,  tendencies  associated  with  the  taking  of  the  survey.  For  example,  it 
was  noted  by  the  team  that  some  surveyees  seemed  to  habitually  favor  the  left 
alternative  in  a  given  series  of  analyses  despite  the  generation  of  inconsistencies.  In 
another  aspect,  it  was  noted  that  some  surveyees  were  extreme  in  their  assessments, 
predominantly  choosing  the  absolute  or  near  absolute  evaluation  of  importance  while 
others  were  neutral.  Such  an  analysis,  especially  in  conjunction  with  the  expert 
system  environment,  could  provide  an  adaptive  survey  instrument  designed  to 
minimize  the  effect  of  such  tendencies. 

5-12.  CONCLUSIONS 

a.  The  survey  process  proved  to  be  an  impressive  mechanism  for  the  acquisition  of 
weighting  factors.  It  was  modular,  adaptable,  insightful,  thought-provoking,  and 
realistic.  The  use  of  the  analytic  hierarchy  process  with  its  pairwise  comparison 
technique  provided  substance  to  the  attempt  to  capture,  quantitatively,  those  aspects 
of  the  program  development  process  which  commonly  defy  expression  in  numerical 
terms.  In  the  course  of  gathering  such  information,  the  Value  Added  team,  as  well  as 
the  survey  participants,  gained  additional  insights  into  how  various  components  of 
the  process  related  to  one  another.  This,  in  and  of  itseif,  was  a  significant 
contribution  of  the  surveys.  Combine  this  insight  with  the  weightings  themselves, 
and  the  resulting  compilation  of  information  is  a  demonstrably  viable  and  useful 
analytic  product. 
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..  b.  The  success  of  the  reception  of  the  results  by  the  survey  population  also 
supports  the  contention  that  the  survey  was  a  viable  medium  for  the  development  of 
weighting  values  as  well  an  introspective  look  into  the  mindset  of  the  survey 
population. 
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CHAPTER 6 

EFFECTIVENESS  INTEGRATION 


6-1.  PURPOSE  AND  USE.  This  chapter  discusses  the  Effectiveness  Integration 
Module  of  the  Value  Added  Analysis.  Figure  6-1  shows  how  the  Effectiveness 
Integration  Module  is  related  to  other  VAA  modules. 


Figure  6-1.  Value  Added  Analysis  Effectiveness  Integration  Module 


a.  Purpose.  The  Effectiveness  Integration  Module  provides  an  organized 
framework  to  integrate  multmle  factors  that  measure  a  weapon  system's  value  into 
one  overall  effectiveness  coefficient.  An  effectiveness  coefficient  is  computed  for  each 
VAA  system;  collectively,  they  provide  a  relative  ranking  of  the  value  of  VAA 
systems. 
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b.  Use.  Effectiveness  coefficients  from  the  Effectiveness  Integration  Module  are 
used  by  the  Optimization  Module  in  its  cost-benefit  tradeoffs  to  determine  the  subset 
of  VAA  systems  that  provides  maximum  effectiveness  for  constrained  resources. 
TOPSIS  is  the  analytical  tool  within  the  Effectiveness  Integration  Module  that 
computes  the  effectiveness  coefficients. 

6-2.  ANALYTICAL  MODELS  ANO  TOOLS- TOPSIS.  VAA  supports  the 
ARSTAF  with  a  quick  turnaround  capability  to  perform  cost-benefit  analysis  of 
competing  alternative  systems  during  the  developmen  t  of  the  Army  POM.  A  subtask 
of  this  support  is  ranking  all  alternative  systems  in  terms  of  their  relative  value.  All 
competing  systems  must  be  conbistently  judged  against  several,  possibly  conflicting, 
criteria.  An  analytical  tool  is  required  to  do  the  ranking  du^  to  the  potentially  large 
numbers  of  alternative  systems,  multiple  decision  criteria,  and  potential  bias  of 
decisionmakers.  Multiple  attribute  decisionmaking  (MADM)  tools  are  well-suited  to 
this  kind  of  problem.  MADM  tools  assist  the  decisionmaker  in  making  logical  and 
consistent  rankings  of  alternatives.  TOPSIS,  a  MADM  tool,  was  used  for 
effectiveness  integration  in  both  Phases  I  and  II  of  VAA.  Several  alternatives  to 
TOPSIS  were  evaluated  during  Phase  I  and  are  documented  in  Chapter  8  of  the  Phase 
I  Report.  TOPSIS  was  selected  based  on  its  popularity  within  the  Army  community, 
its  ease  of  use,  its  simplicity,  and  its  clear  ranking  of  alternatives. 

a.  TOPSIS  Overview.  TOPSIS  was  developed  by  Chaing-Lai  Hwang  and 
Kwangusn  Yoon.  It  can  be  used  to  compare  many  alternatives  that  have  many 
common  attributes  or  criteria,  regardless  of  the  attribute's  units  of  measure.  The 
basic  TOPSIS  problem  is  illustrated  in  Figure  6-2,  where  matrix  rows  represent 
alternatives  (Ai),  columns  repf  eseat  criteria  (Ci),  and  the  weight  associated  with  each 
criteria  is  represented  as  W,.  TOPSIS  is  based  on  the  concept  that  preference  should 
be  given  to  the  alternative  that  is  both  the  shortest  distance  from  the  "ideal"  solution 
(best  alternative)  and  the  farthest  distance  from  the  "negative-ideal"  solution  (worst 
alternative).  Figure  6-3  illustrates  the  concept  using  two  alternatives  Ai  and  A2. 
Notice  that  alternative  Aj  has  shorter  distances  (both  to  the  ideal  solution  A*  and  to 
the  negative-ideal  solution  A-)  than  alternative  A2.  Therefore,  it  is  difficult  to  justify 
selection  of  Ai  over  A2  based  on  the  Euclidian  distances  alone.  TuPSIS  considers  the 
distances  to  both  the  ideal  and  negative-ideal  solution  simultaneously  by  taking  the 
relative  closeness  to  the  ideal  solution. 
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Figure  6-2.  Example  Matrix  for  Basic  TOPSIS 


Figure  6-3.  Euclidean  Distances  to  the  Ideal  and  Negative-ideal  Solution  in 

Two-dimensional  Space 
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b.  TO  PSIS  Algorithm.  For  ease  of  presentation,  the  TOPSIS  method  can  be 
presented  as  a  series  of  operations  on  a  matrix  with  different  alternatives  (rows)  and 
decision  criteria  used  to  judge  the  alternatives  (columns).  The  elements  of  the  matrix 
represent  the  contribution  of  each  alternative  to  achieving  the  criteria.  The 
parameters  of  the  Phase  II  decision  matrix  include  41  alternative  systems  (rows)  and 
eigh.  decision  criteria  associated  with  three  time  periods  and  three  theaters  (8x3x3 
=  72  columns). 

(1)  Inputs  to  TOPSIS.  There  are  two  basic  types  of  inputs  to  TOPSIS— c.ie 
scores,  which  are  elements  of  the  decision  matrix,  and  the  criteria  weights,  since  all 
criteria  are  normally  aot  valued  equally  by  decisionmakers. 

(a)  Scores.  The  scores  in  the  decision  matrix  are  the  values  for  each 
alternative  system's  contribution  to  each  of  the  criteria.  All  VAA  systems  are  not 
available  in  every  theater  and  every  time  period.  For  example,  of  the  41  VAA 
systems,  only  20  are  available  in  the  near-term  (1996),  33  are  available  by  mid-term 
(2001),  and  all  41  are  available  long-term  (2008).  Table  6-1  depicts  the  decision 
matrix  scores  for  the  20  near-term  systems  over  eight  decision  criteria  for  the 
Contingency  Scenario  in  1996.  Decision  matrices  similar  to  Table  6-1  were  developed 
for  each  theater  and  time  period.  The  scores  represent  the  value  of  both  explicit  and 
implicit  factors. 


Table  6-1.  Decision  Matrix  Scores  for  Contingency  96 


Criteria 

Contingency  Scenario  96 

FFK 

BFS 

Political 

risk 

Program- 

mat!'"' 

Asset 

v&n 

O&T 

risk 

Crit  of 

need 

Sustain^ 

ability 

AAWS-M 

2751)683 

.0229967 

5 

3 

8 

4 

8 

6 

ADATS 

2633635 

0167408 

3 

3 

6 

6 

8 

6 

ADVQL'ICKFIX 

.2001821 

0221318 

6 

6 

8 

5 

5 

5 

ARV 

■002206 

-.002277 

5 

4 

(5 

n 

4 

6 

ATAS 

.1703440 

.0113128 

5 

6 

8 

mm 

5 

4 

AVENGERa 

0 

0 

0 

0 

0 

0 

0 

0 

FMTV5 

•148385 

■009207 

6 

7 

8 

9 

7 

7 

GBCSiyH 

.0675732 

.0041592 

5 

5 

7 

7 

6 

6 

HAB 

U  ■'2775 

.0057106 

6 

5 

6 

8 

6 

6 

HEX 

-  03''2O4 

.0022422 

7 

7 

7 

8 

5 

7 

JSTAKS 

1925984 

0137656 

6 

5 

9 

6 

8 

8 

\tlA2 

.058277 

0004247 

3 

3 

6 

6 

4 

5 

SAPXRM 

1195902 

.0106882 

5 

4 

7 

4 

6 

6 

PI.S 

1 9 

01113 

6 

7 

8 

8 

7 

■7 

TOW2B 

■030435 

0007.12 

9 

8 

8 

7 

5 

b 

VEMASID 

000896 

7 

6 

6 

6 

7 

AIR  VOLCANO 

.2073552 

009423 

6 

5 

7 

7 

5 

.5 

VOLCANO 

-  105956 

.002345 

7 

7 

"7 

7 

5 

VVA.M 

4957914 

03573!  1 

5 

7 

8 

5 

FMTV’ 5 

-  148385 

■00920-’ 

*1 

8 

9 

7 

7 

sv.stnm  for  Contins^eru  v  S<  erarii* 
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1..  Explicit  Scores.  Scores  for  the  explicit  factors  (fractional  exchange 
ratio  and  Blue  force  surviving)  are  derived  from  theater  and  time  period  dependent 
combat  simulations  as  discussed  in  Chapter  4. 

2.  Implicit  Scores.  Recall  from  Chapter  5  that  the  six  implicit  factors  are 
(1)  Political  Risk,  (2)  Programmatics,  (3)  Asset  Versatility  and  Deployability,  (4) 
Operational  and  Technological  Risk,  (5)  Criticality  of  Need,  and  (6)  Impact  on 
Sustainability  with  respect  to  combat.  The  score  for  each  alternative  against  the 
implicit  factors  is  provided  by  the  Subject  Matter  Expert  (SME)  Survey.  Refer  to 
Chapter  5  for  definitions  of  implicit  factors  and  details  on  the  SME  Survey.  Scores  of 
a  system  against  implicit  factors  are  not  theater  or  time  period  dependent,  provided 
the  system  is  available  in  the  theater  during  the  period.  For  example,  the  AAWS-M 
is  available  to  all  theaters  and  years;  therefore,  its  implicit  scores  are  the  same  for  all 
theater  and  time  periods.  AVENGER,  on  the  other  hand,  is  not  a  candidate  system  in 
the  Contingency  Scenario  in  1996  because  it  is  already  fielded.  Therefore,  it  receives 
no  value  in  this  scenario. 

(b)  Criteria  Weights.  VAA  criteria  weights  represent  the  relative 
importance  that  Army  decisionmakers  assign  to  the  various  decision  criteria.  The 
criUria  weights  for  VAA  Phase  II  are  summarized  in  Table  6-2.  The  criteria 
weighting  survey  of  Army  senior  leaders  provided  the  data  used  to  compute  criteria 
weights.  Chapter  -5  discusses  the  criteria  weighting  survey  and  the  procedure  ( AHP) 
that  was  used  to  compute  the  weights. 
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Table  6-2.  Summary  of  V  AA  Criteria  Weights 


Scenario  weights  by  theater  and  year: 

Year  Contingency  Stressful 

1996  0.1564  0.0522 

2001  0.1825  0.1037 

2008  0.1082  0.0882 

NEA 

0.1462 

0.1055 

0.0567 

Effectiveness  weights: 

Implicit 

0.2303 

Explicit 

0.7696 

Implicit  criteria  weights: 

Political  risk 

0.0578 

Programmatics 

0.0655 

Asset  versatility  and  deployability 

0.2380 

Operational  and  technical  risk 

0.1734 

Criticality  of  need 

0.3135 

Impact  on  sustainability  WRT  combat 

0.1516 

Explicit  criteria  MOE  weights: 

Fractional  exchange  ratio 

0.4653 

Blue  force  surviving 

0.5346 

(2)  TOPSIS  Procedures.  The  steps  in  the  TOPSIS  algorithm  are  summarized 
below,  in  Figure  6-4,  along  with  the  mathematics  for  each  step.  The  TOPSIS  steps 
are  sequential  operations  that  begin  with  the  initial  decision  matrix.  The  matrix  is 
first  normalized  to  allow  for  comparison  across  criteria  or  columns.  Since  all 
attributes  may  not  have  equal  importance  or  weight,  the  next  step  is  to  construct  the 
weighted  normalized  decision  matrix.  The  third  step  is  to  determine  ideal  and 
negative-ideal  solutions  by  choosing  the  best  available  weighted  normalized  score  for 
each  criteria  and  the  worst  available  weighted  normalized  score  for  each  criteria.  In 
step  4,  the  distance  from  each  alternative  to  both  the  ideal  and  negative-ideal  is 
calculated.  This  step  assumes  that  the  distance  from  the  negative  ideal  is  just  as 
important  to  the  decisionmaker  as  the  distance  to  the  ideal.  Step  5  calculates  the 
relative  closeness  to  the  ideal  solution  or  the  TOPSIS  score,  and  finally  the  preference 
order  is  determined. 
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•  Step  1:  Normalize  Decision  Matrix 

rij  =  normalized  element  of  decision  matrix 


•  Step  2:  Weight  Normalized  Decision  Matrix 

Vij  =  wjrij,  i  =  l,...,nandj  =  l,...,m 

•  Step  3;  Find  Ideal  and  Negative-Ideal  Vectors 


A  * 


max 

=  {  . 

I 


V.. 

y 


i  =  1 


♦  ♦  ♦ 

n}  =  (F  V  ...,V  ) 

1,2,  m 


max 

A  -  =  {  V 

i  V 


■=■ . ) 

1,2,  m 


•  Step  4:  Calculate  Separation  Measures 


*  ,  n  ♦  n1/2 

s  ={  -V  )M 
‘  j  ’ 


7-1 

•  Step  5:  Calculate  TOPSIS  Score 


(V.  -  V 


172 


,  Vi 


S 


‘  *  _  ’ 

s  ,  +s  , 

Figure  6-4.  Stepsof  Basic  TOPSIS 


(3)  TOPSIS  Output.  The  above  procedure  yields  TOPSIS  scores  (coefTicients) 
which  fall  between  zero  (0.0)  and  one  (1.0)  for  each  system.  The  largest  TOPSIS 
coefficient  indicates  the  preferred  alternatives.  TOPSIS  results  are  presented  in 
paragraph  6-3. 
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c.  TOPSIS  Implementation.  Implementation  continued  to  be  one  of  the 
primary  considerations  in  selecting  TOPSIS  for  Phase  II  just  as  it  had  been  in  Phase 
I.  After  the  completion  of  Phase  I,  the  United  States  Military  Academy  (US^IA) 
conducted  an  analysis  of  the  mathematical  foundations  of  TOPSIS.  Based  on  the 
USiVIA  analysis  and  lessons  learned  by  the  Phase  I  study  team,  several  changes  were 
recommended  to  Phase  I  TOPSIS  which  were  implemented  as  a  C-Language  program 
on  a  PC. 

(1)  Phase  II  Implementation.  In  order  to  provide  timely  Phase  II  support  to 
building  the  FY  94  -  FY  99  POM,  Phase  II  TOPSIS  was  implemented  as  spreadsheets. 
Spreadsheet  implementation  was  considered  more  timely  than  modifying  the  C- 
Language  program  to  evaluate  recommended  changes  to  Phase  I  TOPSIS  which 
included: 

(a)  Use  fixed  upper  and  lower  bounds  when  establishing  the  ideal  and 
negative-ideal  to  preclude  rank  order  changes  (USMA  suggestion). 

(b)  Avoid  washing  out  implicit  factors. 

L  Compute  TOPSIS  results  for  explicit  and  implicit  factors  separately  and 
then  combine  them  using  their  respective  weights. 

2.  Do  not  normalize  the  implicit  factors  which  represent  a  common  unit  of 

measure. 

(2)  Implementation  Environment.  Phase  II  TOPSIS  was  initially 
implemented  on  the  personal  computer  and  subsequently  on  the  METAPHOR  data 
base  computer  as  contract  support  became  available. 

(a)  PC  TOPSIS.  PC  TOPSIS  is  implemented  as  multiple  linked 
spreadsheets  using  WINDOWS  3.0  EXCEL  software  and  IBM  PC  compatible 
hardware.  Spreadsheets  are  good  for  analyzing  the  effect  of  changes  in  input  data, 
provided  that  the  operational  and  logical  procedures  remain  fairly  stable.  Liberal 
use  of  macros  ensures  responsiveness  of  PC  TOPSIS.  However,  a  limitation  is  that 
the  spreadsheets  may  become  large  and  unwieldy. 

(b)  M  ETA  PHO  K  TO  PS  IS.  TOPSIS  was  implemented  on  the  METAPHOR 
data  base  computer  which  is  located  in  the  Pentagon  with  terminal  links  at  CAA. 
Steps  were  taken  to  ensure  that  METAPHOR  TOPSIS  uses  the  appropriate  logic  and 
produces  the  expected  results.  Documentation  of  METAPHOR  TOPSIS,  a  module  of 
the  VAA  METAPHOR  Desktop,  will  be  covered  in  a  separate  document. 
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6-3.  RESULTS.  This  paragraph  presents  some  major  assumptions  governing 
computation  of  TOPSIS  coemcients,  presents  the  computed  TOPSIS  coefficients,  and 
discusses  limitation  on  the  use  of  the  coefficients. 

a.  Assumptions 

(1)  An  alternative  system  receives  no  implicit  value  if  it  is  not  available  to  the 
theater  in  the  specified  time  period. 

(2)  Decisionmakers  are  risk-neutral  and  consider  the  distance  from  the  worst 
alternative  to  be  just  as  important  as  the  distance  to  the  best  alternative  in  ranking 
the  alternatives. 

b.  Computed  VAA  Coefficients.  Relative  overall  measures  of  worth,  normally 
referred  to  as  VAA  coefficients,  were  computed  for  each  alternative  system  by  theater 
as  well  as  across  theaters  to  obtain  a  global  view.  Within  a  theater  or  multitheater 
scenario,  separate  views  of  the  coefficients  were  maintained  by  year  and  by  type  of 
factors,  i.e.,  based  on  implicit  factors  only,  explicit  factors  only,  or  a  combined  view  of 
implicit  and  explicit  factors.  The  coefficients  for  the  Contingency  Scenario  are 
presented  in  Tables  6-3,  6-4,  and  6-5.  Tables  6-6,  6-7,  and  6-8  present  similar  results 
for  the  Stressful  Scenario.  The  results  for  the  global  or  multi  theater  view 
(Contingency  and  Stressful  combined)  are  shown  in  Tables  6-9,  6-10,  and  6-11. 


6-9 


CAA-SR-92-10 


Table  6-3.  VAA  System  Coefficients  for  Contingency  Scenario 
_ _ (implicit  only) _ _ 


1996 

2001 

2008 

System  name 

Coefficient 

System  name 

Coefficient 

System  name 

Coefficient 

FMTV5 

.759 

FMTV5 

.764 

FMTV5 

.765 

FMTV2.5 

.759 

F.MTV2.5 

.764 

FMTV2.5 

.765 

JSTARS 

.741 

PLS 

.740 

PLS 

.740 

PLS 

.737 

JSTARS 

.729 

JSTARS 

.724 

AAWS-M 

.584 

RAH  66 

.695 

RAH-66 

.690 

ADATS 

.579 

STINGRAY 

.632 

STINGRAY 

.628 

VEMASID 

547 

LOSAT 

.597 

LOSAT 

.592 

HAB 

.497 

ADV  FF 

.576 

ADVFF 

.577 

GN'D  VOLCANO 

.490 

ADATS 

.568 

ADATS 

.563 

GBCS-LH 

.490 

AAWS-M 

.567 

AAWS-M 

.561 

HET 

.471 

ATACMS  II 

.548 

RAH-66-LB 

.550 

TOW2B 

.463 

VEMASID 

.547 

VEMASID 

.546 

MRS SADARM 

384 

AGS 

.514 

ATACMS  II 

544 

AIR  VOLCANO 

.356 

AH-64LB 

.514 

AH-64  LB 

.512 

WAM 

.341 

HAB 

.507 

AGS 

.510 

ATAS 

.338 

GND  VOLCANO 

.504 

CMV 

.510 

ADVQLICKFLX 

.331 

GBCS-IVH 

.495 

HAB 

.509 

ARV 

.278 

GBS 

.495 

GND VOLCANO 

.506 

M1A2 

.177 

HET 

.490 

GBCS-GU 

.496 

TOW2B 

.484 

GBS 

.496 

NLOS-AT 

.466 

HET 

.495 

MLRS SADARM 

.379 

TOW2B 

.489 

AIR  VOLCA.no 

.372 

ADVSEMA 

.482 

ATAS 

.353 

NLOS-AT 

.461 

WA.M 

.353 

BLOCK  III 

.433 

ADVQL  ICKFI.X 

344 

FARV-A 

.429 

MLRS  TGW 

.333 

MLRS SADARM 

.376 

155LTWT 

.292 

airvolca.no 

,375 

ARV 

.292 

ATAS 

,356 

155  ER 

.276 

WAM 

.355 

AMS-H 

.249 

ADVQLTCKFIX 

.346 

155  TGP 

.230 

AFAS 

334 

M1A2 

.187 

MLRS  TGW 

.330 

ARV 

.296 

155  I.T  WT 

295 

1.55  ER 

.278 

AMS-H 

251 

I55T(3P 

231 

M 1  A2 

.189 

FIFV 

161 

AVENGER 

1) 
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Table  6-4.  VAA  System  Coefficients  for  Contingency  Scenario 


(explicitonly) 


1996 

2001 

2008 

System  name 

Coefficient 

System  name 

Coefficient 

System  name 

Coefficient 

W.AM 

1 

A  DATS 

l 

RAH-66  LB 

1 

.\AWS-M 

.689 

MLRS  TGVV 

.791 

RAH-66 

.939 

.•KDVQUICKFIX 

M1A2 

.635 

MLRS  TGW 

.799 

.^D.ATS 

AH.64LB 

.537 

FIFV 

.780 

.JST.XRS 

.519 

HAB 

.524 

ADATS 

.691 

.AIRVOLCA.N’O 

.477 

WAM 

.513 

GBCS-Lj'H 

.639 

ATAS 

.474 

GBS 

.497 

AH  64  LB 

.625 

MLRS  SADAR.M 

.431 

ATAS 

.481 

AF.AS 

.591 

PLS 

.400 

RAH-66 

.455 

FMTV5 

.589 

HAB 

.400 

.AT. AC  MS  U 

.394 

FMTV2.5 

.589 

GBCS-IVH 

.315 

155  TGP 

.385 

AIR  VOLCANO 

.565 

HEX 

.220 

155  LTVVT 

.363 

155  TGP 

.535 

ARV 

.190 

ARV 

.322 

HEX 

.519 

TOW2B 

.186 

HEX 

.316 

ADVQLTCKFIX 

.503 

■  448 
.427 
.425 
.425 
.420 
.409 
.398 
364 
349 
.344 
.343 
.313 
.290 
289 
.285 
280 
261 
.257 
.255 
217 
214 
191 
167 
167 
.138 
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Table  6-5.  VAA  System  Coefficients  for  Contingency  Scenario 
_ (combined  implicit  and  explicit) _ 


1996 

2001 

2008 

System  name 

Coefficient 

System  name 

Coefficient 

System  name 

Coefficient 

VVAM 

ADATS 

.984 

RAH-66  LB 

.985 

AAWS-M 

.689 

MLRSTGW 

.790 

RAH-66 

.938 

ADVQUICKFIX 

623 

M1.A2 

.635 

MLRS  TGW' 

.798 

ADATS 

.605 

AH-64  LB 

.537 

FIFV 

.779 

JSTARS 

.520 

HAB 

.524 

ADATS 

.691 

AIR  VOLCANO 

.477 

WA.VI 

513 

GBCS-UH 

.639 

ATAS 

,474 

GBS 

497 

AH-64  LB 

.625 

MLRS SADARM 

431 

ATAS 

.481 

AFAS 

.591 

PLS 

.401 

RAH-66 

456 

FMTV5 

.589 

HAB 

.400 

ATACMSII 

.394 

FMTV2.5 

.589 

GBCS-I7H 

315 

155  TGP 

.385 

AIR  VOLCANO 

.565 

HET 

221 

155  LT  WT 

.363 

1 55  TGP 

.535 

ARV 

.190 

ARV 

.322 

HET 

.519 

TOW2B 

.187 

HET 

.317 

ADVQLTCKFIX 

.503 

M1A2 

,184 

ADVFF 

.315 

FARV-A 

.448 

VEMASID 

.164 

AAWS-M 

.257 

STINGRAY 

.427 

GND  VOLCANO 

,122 

155  ER 

251 

VVAM 

.425 

FMTV5 

.028 

STINGRAY 

,247 

TOW2B 

.425 

FMTV2.5 

.028 

JSTARS 

.245 

HAB 

.420 

FMTV5 

.243 

155LTVVT 

.409 

FMTV2.5 

.243 

MLRSSADARM 

.398 

GND VOLCANO 

240 

VEMASID 

.364 

GBCS-L'H 

.231 

155  ER 

.349 

N  LOS- AT 

.220 

JSTARS 

345 

AIR  VOLCANO 

,212 

GBS 

.343 

VE.MASID 

.211 

GND  VOLCANO 

.314 

TOVV2B 

.206 

ADV  FF 

.291 

PLS 

203 

ATAS 

.289 

LOS  AT 

.202 

AAWSM 

.286 

AMS-H 

.183 

PLS 

.281 

ADVqLTCKFIX 

.174 

ADV  SKM  A 

.261 

MLRS SADARM 

162 

AMS  H 

.257 

AGS 

.017 

N LOS  AT 

256 

I. OS. AT 

.218 

ATACMS  II 

.215 

ARV 

.191 

CMV 

.168 

AGS 

.168 

BI.Oi'K  III 

139 

Ml  A2 

007 

AVENGER 

0 
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Table  6-6.  VAA  System  Coefficients  for  Stressful  Scenario 
_ _ (implicit  only) _ _ 


1996 

2001 

2008 

System  name 

Coefficient 

System  name 

Coefficient 

System  name 

Coefficient 

FMTV5 

.760 

FMTV5 

.764 

FMTV5 

.765 

FMTV2.5 

.760 

FMTV2.5 

.764 

FMTV2.5 

.765 

JSTARS 

743 

PLS 

.741 

PLS 

.741 

PLS 

736 

JSTARS 

.731 

JSTARS 

.727 

AAWS-M 

.587 

RAH-66 

.698 

RAH-66 

.693 

ADA  IS 

583 

STINGRAY 

.636 

STINGRAY 

.631 

VEMASID 

549 

LOSAT 

.601 

LOSAT 

.596 

HAB 

499 

ADVFF 

.575 

ADV  FF 

.576 

GBCS-L/H 

490 

A  D  ATS 

.573 

ADATS 

.567 

GND  VOLCANO 

.490 

AAWS-M 

.5  72 

AAWS  M 

.565 

AVENGER 

.486 

ATACMSII 

.550 

RAH-66  LB 

.553 

HET 

469 

VEMASID 

.549 

VEMASID 

.54" 

T0W2B 

460 

AH-64  LB 

514 

ATACMSII 

,546 

MLRSSADARM 

386 

HAB 

.507 

CMV 

,513 

AIR  VOLCANO 

354 

GND  VOLCANO 

.503 

AH-64  LB 

.512 

WAM 

.339 

AVENGER 

.502 

HAB 

.508 

ATAS 

,336 

GBCS-IMI 

.496 

AVENGER 

.507 

ADV  QL  ICKFtX 

328 

(}BS 

.496 

GND  VOLCANO 

,505 

ARV 

.277 

HET 

.486 

GBCS-UH 

.496 

M1A2 

177 

TOW2B 

.479 

GBS 

496 

N  LOS- AT 

.470 

HET 

.492 

MLRSSADARM 

381 

TOW2B 

.485 

AIR  VOLCANO 

.370 

ADVSEMA 

.483 

ATAS 

.351 

NLOS-AT 

.465 

WAM 

.351 

BLOCK  III 

.437 

ADVQLTCKFIX 

.341 

FARVA 

.427 

MLRS  TGW 

.336 

MLRSSADAR.M 

.3"S 

ARV 

.290 

AIR  VOLCANO 

,373 

1.55  LT  WT 

.290 

ATAS 

354 

155  ER 

.274 

WAM 

353 

AMS-H 

248 

ADV  QCICKFIX 

344 

155  FGP 

229 

A  F  AS 

,3.36 

MIA2 

186 

M1.KML.W 

.332 

ARV 

.294 

155  LT  U’l 

.293 

1 55  EK 

2"6 

AMS-H 

,249 

155  TGP 

.230 

M1.A2 

188 

FIFV 

.163 
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Table  6-7.  VAA  System  Coefficients  for  Stressful  Scenario  (explicit  only) 


1996 

2001 

2008 

System  name 

Coefficient 

System  name 

Coefficient 

System  name 

Coefficient 

AAWS-M 

.624 

RAH-66 

.731 

FIFV 

1 

HAB 

.609 

GBS 

.590 

AH-64  LB 

.548 

WAM 

.594 

HAB 

.561 

BLOCK  III 

.465 

TOW2B 

.530 

M1A2 

.557 

RAH-66  LB 

.463 

VEMASID 

.520 

ADV  FF 

.554 

AAWS-M 

.444 

M1A2 

.508 

NLOS-AT 

.521 

RAH. 66 

.429 

GND VOLCANO 

.471 

15.5  ER 

.514 

GBS 

.429 

FMTV5 

.467 

.AH.64LB 

.497 

TOW2B 

.410 

FMTV2.5 

.467 

WA.Vl 

.481 

M1.A2 

.409 

ARV 

.466 

AMS-H 

.457 

HAB 

.385 

AIR  VOLCANO 

.464 

GBCS-LH 

453 

PLS 

.381 

HET 

.439 

JSTARS 

.418 

WAM 

352 

ADATS 

430 

ARV 

.399 

JSTARS 

.351 

PLS 

.348 

AVENGER 

.388 

ADATS 

313 

ATAS 

.330 

MLRSSADARM 

.344 

ATACMSII 

310 

JSTARS 

.322 

PLS 

.341 

VEMASID 

.287 

ADVQLTCKFIX 

.321 

TOW2B 

.340 

155  LTWT 

.287 

AVENGER 

.265 

ADVQLTCKFiX 

.337 

AMS-H 

.283 

MLRSSADARM 


GBCS-ITH 


AlKVULl.  A.NU 


ARV 


FARV  A 


ADVSEMA 


HET 


GBCSt.H 


FMTV5 


FMTV2.5 


M.OS  AT 


(}ND  VOl.r  ANO 


I.OSAT 


155  TOP 


MEKSSADARM 


ADVCJLirKFIX 
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Table  6-8.  VAA  System  Coefficients  for  Stressful  Scenario 
_ (combined  implicit  and  explicit ) _ 


1996 

2001 

2008 

System  name 

- 1 

Coefficient 

System  name 

Coefficient 

System  name 

Coefficient 

AAWS-M 

.624 

RAH-66 

.731 

FIFV 

.979 

HAB 

.609 

GBS 

.589 

AH-64  LB 

.547 

WAM 

.594 

HAB 

.561 

BLOCK  III 

.465 

T0VV2B 

.530 

M1A2 

.557 

RAH-66  LB 

.463 

VEMASID 

.520 

ADVFF 

.554 

AAWS-M 

.444 

M1A2 

.508 

NLOS-AT 

.521 

RAH-66 

.429 

gndvolca.no 

.471 

1.55  ER 

.514 

GBS 

.429 

FMTV5 

.467 

AH-64  LB 

.497' 

XOW2B 

.410 

FMTV2.5 

.467 

WAM 

.481 

M1A2 

.409 

ARV 

.466 

AMS-H 

.456 

HAB 

.385 

AIR  VOLCANO 

.464 

GBCS-LXI 

.453 

PLS 

.381 

HEX 

.437 

■JSTARS 

418 

WAM 

.352 

ADATS 

.428 

ARV 

.399 

.JSTARS 

.352 

PLS 

.349 

AVENGER 

388 

ADAXS 

.313 

ATAS 

.330 

MLRS SADARM 

.344 

AXACMSII 

.310 

JSTARS 

.323 

PLS 

.341 

VEMASID 

.288 

ADVQUICKFIX 

.321 

TOVV2B 

.340 

155LXWX 

.287 

AVENGER 

.266 

ADVQLXCKFIX 

.337 

AMS-H 

.283 

MLRSSADARM 

.172 

HEX 

.310 

STINGRAY 

.281 

GBCS-IVH 

.024 

AXAS 

292 

AVENGER 

.280 

ADAXS 

.283 

155  ER 

.280 

LOSAX 

.278 

AFAS 

.263 

AAWS-M 

.277 

ADV  FF 

.256 

MLRS  XGW 

.273 

MLRS  XGW 

.232 

GND VOLCANO 

.270 

AIR  VOLCANO 

.231 

STINGRAY 

.239 

ARV 

229 

AXACMSII 

.203 

FARV-A 

.228 

155LXWX 

.182 

ADVSEMA 

.210 

AIR  VOLCANO 

.165 

HEX 

.209 

FMXV5 

119 

GBCS-LXI 

.207 

FMXV  2.5 

.119 

FMXV5 

.174 

VEMASID 

104 

FMTV2.5 

.174 

15.5  TOP 

090 

NLOS-.AX 

.165 

GND  VOLCANO 

.153 

LOSAX 

.125 

CMV 

■IDHH 

AXAS 

155  XGP 

.095 

MLRSSADARM 

.094 

ADV  Ql  ICKFIX 

.068 
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Table  6-9.  VAA  System  Coefficients  for  Contingency  and  Stressful 
_ Scenarios  (combined  implicit) _ 


1996 

2001 

2008 

System  name 

Coefficient 

System  name 

Coefficient 

System  name 

Coefficient 

FMTV5 

.759 

FMTV5 

.764 

FMTV5 

.765 

FMTV2.5 

.759 

FMTV2,5 

.764 

FMTV2.5 

.765 

JSTARS 

.741 

PLS 

.740 

PLS 

.741 

PLS 

.737 

JSTARS 

.729 

•JSTARS 

.725 

AAWS-M 

.584 

RAH-66 

.696 

RAH  66 

.692 

ADATS 

.579 

STINGRAY 

.633 

STINGRAY 

.630 

VEMASID 

.547 

LOSAT 

.598 

LOSAT 

.594 

HAB 

.498 

ADVFF 

.576 

ADV  FF 

,577 

GND  VOLCANO 

490 

ADATS 

.569 

ADATS 

GBCS-IVH 

.490 

AAVVS-M 

.568 

AAWS-M 

.563 

AVENGER 

486 

ATACMSII 

.548 

RAH-66  LB 

.551 

HET 

.471 

VEMASID 

.548 

VEMASID 

546 

TOW2B 

463 

AGS 

514 

ATACMSII 

544 

MLRS  SADARNt 

.384 

AH  64  LB 

.514 

AH-64LB 

512 

AIR  VOLCANO 

.355 

HAB 

.507 

CMV 

.511 

WAM 

.340 

GND  VOLCANO 

.504 

AGS 

.510 

ATAS 

.338 

AVENGER 

.502 

HAB 

.508 

ADVQCICKFIX 

.330 

GB(.S-UH 

.496 

AVENGER 

.507 

ARV 

.278 

GBS 

.496 

GND VOLCANO 

.506 

M1A2 

.177 

HET 

.489 

GBCS-L/H 

.496 

TOW2B 

.483 

GBS 

.496 

N  LOS- AT 

.467 

HET 

,494 

MLRS SADARM 

.380 

TOW2B 

.487 

AIR  VOLCANO 

.371 

ADVSEMA 

.483 

AT.AS 

.353 

N  LOS- AT 

.463 

WAM 

.352 

Bl.OCK  III 

.435 

ADVqUlCKFIX 

.343 

FARVA 

427 

MLRS  TGW 

.334 

MLRS SADARM 

,377 

155  LTWT 

292 

AIR  VOLCANO 

.374 

ARV 

292 

ATAS 

.355 

155  ER 

.275 

WAM 

.354 

AMS-H 

.249 

ADVQLTCKFIX 

,345 

155  TGP 

230 

AF.AS 

.334 

M1A2 

187 

MLRS  TGW 

.331 

ARV 

.295 

155  LT  WT- 

.294 

155  ER 

.277 

AMS  H 

.250 

155  TGP 

230 

Ml. A  2 

189 

FIFV 

162 

6-16 


CAA-SR-92-10 


Table  6-10.  VAA  System  Coefficients  for  Contingency  and  Stressful 

Scenarios  (combined  explicit) 


System  name  I  Coefficient 
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Table  6-11.  VAA  System  Coefficients  for  Contingency  and  Stressful 
_ Scenarios  (combined  implicit  and  explicit) _ 


1996 

2001 

2008 

System  name 

Coefficient 

System  name 

Coefficient 

System  name 

Coefficient 

wa:.. 

.837 

ADATS 

.712 

FIFV 

.855 

AAWS-M 

.678 

MLRS  TGW 

.637 

RAH-66  LB 

.653 

ADVQUICKFIX 

.578 

.Ml. A  2 

.615 

RAH-66 

631 

ADATS 

.575 

HAB 

.536 

AH-64LB 

.583 

JSTARS 

.492 

RAH-66 

.534 

MLRS  TGW 

.498 

AIR  VOLCANO 

.475 

AH  64  LB 

.527 

ADATS 

.493 

ATAS 

.454 

GBS 

524 

AFAS 

.425 

HAB 

.442 

W.AM 

.505 

GBCS-L/H 

,425 

MLRSSADARM 

395 

AT.AS 

435 

TOW2B 

116 

PLS 

393 

ADV  FF 

388 

HAE 

.401 

GBCSUH 

.285 

AVENGER 

,388 

AIR  VOLCANO 

.398 

HET 

.273 

ATACMSII 

.351 

FMTV5 

389 

rOW2B 

.269 

ARV 

.343 

FMTV2  5 

.389 

AVENGER 

,266 

155  ER 

.339 

GBS 

.388 

ARV 

,258 

155  TGP 

.327 

WA.M 

.387 

M1A2 

.247 

NLOS-AT 

.325 

AAWS-M 

.376 

VEMASID 

242 

155LT  WT 

324 

HET 

.368 

GND'.OLCANO 

,2U 

HET 

.315 

STINGRAY 

,352 

FMTV5 

.182 

GBCS-LVH 

307 

JSTARS 

348 

FMTV2  5 

.182 

.)  STARS 

301 

155  LT  WT 

,347 

AMS-H 

.272 

FARVA 

,340 

AAWS-M 

.262 

155  TGP 

.337 

TOW2B 

249 

PLS 

.336 

GND  VOLCANO 

.247 

BLOCK  III 

332 

STINGRAY 

245 

VEMASID 

324 

PLS 

.24! 

155  ER 

313 

LOSAT 

22’2 

ADVQUICKFIX 

.312 

ADVQUICKFIX 

.220 

AVENGER 

280 

F.MTV5 

217 

M1A2 

277 

FMTV2.5 

217 

ADVFF 

273 

MLRSSADARM 

216 

AMS-H 

271 

AIR  VOLCANO 

.202 

ATACMS  II 

.269 

VEMASID 

189 

MLRSSADARM 

.262 

AGS 

.017 

GND VOLCANO 

.238 

ADVSEMA 

235 

ARV 

212 

NLOr  .-aT 

211 

ATAS 

.201 

LOSAT 

174 

AGS 

,168 

CMV 

142 
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c.  Limitations  of  Coefficients.  TOPSIS  coefficients  are  used  as  measures  of 
system  benefit  in  the  Optimization  Module.  This  use  has  been  questioned.  It  is  felt  by 
some  that  while  TOPSIS  coefficients  represent  relative  ranking  among  the 
alternatives,  they  do  not  necessarily  provide  measures  of  the  magnitude  of  difference 
between  alternatives.  F urther  study  of  this  area  is  ongoing. 

6-4.  SENSITIVITY  ANALYSIS.  Two  areas  of  analysis  evaluated  the  effect  of 
changes  in  criteria  weights  on  computed  system  coefficients.  The  first  area  examined 
the  sensitivity  of  computed  coefficients  to  effectiveness  criteria  weights  only.  The 
second  area  looked  at  the  sensitivity  of  computed  coefficients  to  all  criteria  weights. 

In  the  latter  case,  coefficients  were  computed  with  and  without  weights  and 
compared  for  discernible  differences. 

a.  Sensitivity  of  Computed  Coefficients  to  Effectiveness  Weights  Only. 

VAA  coefficients  appear  to  be  highly  correlated  with  the  explicit  effectiveness 
factors.  One  might  anticipate  this,  since  the  surveyed  decisionmakers  assigned 
greater  weight  to  explicit  measures  (0.767)  than  to  implicit  measures  (0.233). 

(1)  Methodology.  To  assess  the  sensitivity  of  VAA  coefficients  to  changes  in 
the  effectiveness  weights,  the  ratio  of  implicit-to-explicit  weight  was  allowed  to  vary 
while  all  other  criteria  weights  were  held  constant.  The  effectiveness  weight  ratio 
was  varied  by  successively  adding  10  percent  to  the  base  implicit  weight  (0.23)  and 
subtracting  10  percent  from  the  base  explicit  weight  (0.77).  Effectiveness  weights 
that  were  evaluated  in  the  sensitivity  analysis  ranged  from  the  base  implicit-to- 
explicit  ratio  of  0.233/0.767  to  a  ratio  of  0.93/0.07. 

(2)  Results  and  Observations.  Figure  6-5  shows  system  coefficients  that 
were  computed  for  selected  effectiveness  weight  ratios.  In  general,  effectiveness 
weights  appear  to  affect  the  coefficients.  However,  the  coefficients  do  not  appear  very 
sensitive  to  small  changes  in  implicit  weights.  Some  specific  observations  follow. 
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Figure  6-5.  Variation  in  the  1996  Contingency  Scenario  Coefficients  as 

Effectiveness  Weights 
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(a)  The  rank  ordering  of  the  20  systems  did  not  change  when  the  base 
efTectiver  iss  ratio  was  changed  by  10  percent.  There  were  slight  changes  in  the 
coefficients  computed  for  only  a  few  systems,  namely,  the  family  of  medium  tactical 
vehicles  (FMTV)  (5-ton  and  2.5-ton)  trucks  and  the  WAM  system. 

(b)  The  rank  ordering  of  the  20  systems  changed  when  the  ratio  of 
effectiveness  weights  changed  by  50  percent.  In  this  case,  the  effectiveness  weights 
were  reversed  (i.e.,  the  implicit  weight  became  0.737  and  the  explicit  weight  became 
0.233).  Actual  changes  in  the  coefficients  were  still  relatively  small  for  most  systems. 
The  larger  changes  occurred  for  top  rated  WAM,  whose  coefficient  decreased  to  0.82 
and  for  the  FMTV,  whose  coefficient  increased  almost  tenfold  (from  0.028  to  0.21). 

(c)  Systems  such  as  the  Air  Defense  Antitank  System  (ADATS),  M1A2 
Abrams  main  battle  tank,  and  SADARM  appear  basically  unaffected  by  changes  in 
implicit  factor  weights,  whereas  the  value  of  FMTVs  appear  to  increase  as  the  weight 
of  implicit  factors  increased. 

(d)  Changes  in  system  coefficients  became  statistically  discernible  at  the 
extreme  of  the  ratio  range  (i.e.,  implicit  is  0.093  and  explicit  is  0.07)  for  most  systems 
at  an  80  percent  confidence  level. 

b.  Sensitivity  of  Computed  Coefficients  to  All  Criteria  Weights.  There  are 
references  in  the  literature  that  suggest  criteria  weights  may  have  little  effect  on  the 
coefficients  computed  with  multiattribute  utility  models.  This  premise  was 
evaluated  for  selected  Phase  II  results. 

(1)  Methodology.  System  coefficients  were  computed  for  the  Contingency 
Scenario  96  with  all  criteria  weights  set  to  1.0  and  compared  to  the  base  case 
Contingency  Scenario  96  coefficients  that  were  computed  with  the  survey  criteria 
weights.  Since  criteria  weights  are  scaler  multipliers,  setting  all  weights  to  1  is 
equivalent  to  computing  the  coefficients  without  weights. 

(2)  Results  and  Observations.  Table  6-12  compares  the  system  coefficients 
computed  without  weights  to  the  base  case  coefficients  computed  with  survey 
weights.  Observations  from  this  comparison  are: 

(a)  The  relative  overall  ranking  of  systems  changed  when  weights  were  not 
used.  However,  the  seven  top  ranked  systems  were  the  same  for  the  base  case  and  the 
”no  weights”  case.  FMTVs  and  a  few  other  systems  moved  up  slightly  in  rank. 

(b)  There  were  discernible  differences  in  the  coefficients  computed  with  and 
without  weights  for  individual  systems  as  shown  in  Table  6-12.  While  the  differences 
do  not  appear  to  be  statistically  significant  based  on  standard  parametric  and 
nonparametric  tests,  the  data  observations  suggest  that  weights  do  have  an  effect  on 
rank  order.  The  test  of  hypothesis  about  difference  in  means  was  used  as  the 
parametric  test  of  significance.  Nonparametric  tests  used  were  the  Sign  test  and  the 
Wilcoxon  Matched-Pairs  Signed-Rank  Test. 
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Rank 

Base  case 

No  weights 

System  name 

Coefficients 

System  name 

Coefficients 

1 

WAM 

.976 

WAM 

.819 

2 

AAWS-xM 

.689 

AAWS-M 

.624 

3 

ADVQUICKFIX 

.623 

ADV  QUICKFIX 

.586 

4 

ADATS 

.605 

ADATS 

.557 

5 

JSTARS 

520 

JSTARS 

.531 

6 

AIR  VOLCAxNTO 

.477 

AIR  VOLCAxNO 

.478 

7 

ATAS 

.474 

ATAS 

.464 

8 

SADARM 

431 

PLS 

.433 

9 

PLS 

.401 

HAB 

.423 

10 

HAB 

.400 

SADARM 

.413 

11 

GBCS-L/H 

.315 

GBCS-L/H 

.334 

12 

hb:t 

.221 

TOW2B 

.305 

13 

ARV 

190 

HET 

.298 

14 

TOVV2B 

.187 

VEMASID 

.243 

15 

M1A2 

.184 

VOLCANO 

.226 

16 

VEMASID 

164 

FMTV5 

.220 

17 

VOLCANO 

122 

FMTV2  5 

.220 

18 

FMTV5 

.028 

ARV 

.220 

19 

FMTV2  5 

028 

M1A2 

.177 
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6-5.  FUTURE  DEVELOPMENT.  Future  developments  for  the  Effectiveness 
Integration  Module  will  study  the  apparent  insensitivity  of  overall  system 
coefficients  to  effectiveness  weights.  There  will  be  further  evaluation  of  alternatives 
to  the  TOPSIS  tool.  Currently,  the  AHP  is  used  to  compute  criteria  weights,  and 
TOPSIS  is  used  to  combine  the  system  criteria  scores  into  an  overall  system  score 
that  is  weighted  by  the  criteria  weights.  Expert  Choice  Software  for  AHP's  rating 
modules  is  a  recommended  approach  when  the  number  of  alternatives  is  too  many  for 
effective  pairwise  comparisons. 

6-6.  SUMMARY'.  TOPSIS  is  a  MADM  tool  that  is  popular  within  the  Army.  VAA 
used  TOPSIS  to  integrate  the  values  that  a  system  contributes  to  the  fulfillment  of 
the  various  decision  criteria  into  an  overall  measure  of  the  system's  benefit.  These 
overall  measures  are  highly  correlated  with  the  explicit  measures  of  effectiveness 
derived  from  the  combat  simulations.  This  correlation  coincides  with  the  expectation 
extrapolated  from  decisionmaker  surveys.  The  strong  correlation  observed  between 
the  overall  coefficients  and  the  combat  simulated  MOE  suggests  that  the  coefficients 
can  be  used  with  some  confidence  despite  the  limitation  discussed  in  paragraph  6-3c. 
Furthermore,  the  sensitivity  analysis  that  was  conducted  suggests  that  the 
coefficients  do  not  fluctuate  greatly  for  small  changes  in  the  inputs. 
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CHAPTER? 

COSTING 


7-1.  INTRODUCTION 

a.  The  purpose  of  the  costing  process  is  to  provide  accurate  cost  input  to  the  Value 
Added  Analysis  optimization  model  that  will  result  in  reliable  cost  outputs  for  a  wide 
range  of  applications.  Weapon  system  cost-quantity  relationships,  weapon  system 
categorizations,  impacts  on  appropriations,  and  formulation  of  budget  constrains 
applicable  to  VAA  systems  were  considered  in  the  costing  process.  Figure  7-1 
provides  an  overview  of  the  costing  process. 


Figure  7- 


b.  Baseline  RDTE  and  procurement  costs  for  VAA  systems  are  provided  as  input 
to  the  VAA  optimization  model.  With  respect  to  procurement  costs,  systems  are 
categorized  as  to  whether  cost-quantity  curves  (e.g.,  learning  curve)  are  used.  First 
unit  costs  (FUCs)  and  cost-quantity  curves  are  provided  for  those  systems  for  which 
this  data  was  available  and  applicable.  ("Cost-quantity"  reflects  economies  of  scale 
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in  terms  of  materials  and  labor  as  opposed  to  exclusive  "learning”  on  the  part  of  the 
production  labor  force.)  Average  unit  procurement  costs  are  used  for  the  remaining 
systems.  Budget  constraints  are  determined  by  applying  a  ratio,  representing  the 
proportion  of  Value  Added  systems  to  th''-  i-dal  RDA  budget,  to  a  fixed  total 
obligational  authority  provided  by  the  sponsor. 

c.  Cost-quantity  relationships  are  dynamically  computed  in  the  VAA  opti¬ 
mization.  A  detailed  discussion  of  this  process  is  provided  in  Chapter  8. 

d.  Life  cycle  costs  are  based  on  the  baseline  cost  estimate  P-92  cost  code  structure. 
This  breakdown  allows  for  computation  of  weapon  system  and  appropriation 
categories. 

7-2.  COST  DATA  SOURCES 

a.  Complete  life  cycle  cost  data  were  requested  from  the  US  Army  Cost  and 
Economic  Analysis  Center  (CEAC)  for  all  VAA  weapon  systems  to  use  for  the  P-92 
cost  categories  used  for  the  standard  VAA  LCCM  template  shown  in  Appendix  E. 
These  data  were  extracted  from  the  executive  summary  portions  of  the  BCE  or  Army 
cost  position  (ACP),  whichever  provided  the  most  current  data.  BCE/ACP  data  were 
not  available  for  those  developmental  systems  that  did  not  reach  Milestone  2 
(Development  Approval),  at  which  time  a  BCE  is  required.  Systems  at  Milestones  0 
and  1  generally  had  only  RDTE  and  procurement  cost  estimates.  Figure  7-2 
illustrates  the  acquisition  milestones  and  phases  for  a  weapon  system.  Table  7-1 
defines  the  cost  categories  for  the  LCCM. 


Program  start 


As  'eaui^ea 


Figure  7-2.  Acquisition  Milestones  and  Phases 
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Table  7-1.  Major  Weapon  System  Cost  Code  Definitions 


Major  weapon  system  definition 

LCCM  cost  elements 

Flyaway  cost 

Manufacturing 

Engineering 

Tooling 

Quality  control 

Nonrecurring  "startup" 

Allowance  for  changes 

Other 

Flyaway  cost 

2.021  Manufacturing 

2.022  Rec  engineering 

2.023  Tooling 

2.024  Quality  control 

2.01  Nonrecurring 

2.03  Engineering  changes 

2.05  System  test  &  evaluation 

2.091  System  project  management 

2.094  Other  production  cost 

Weapon  system  cost 

Flyaway  cost 

Technical  data 

Contractor  services 

Support  equipment 

Training  equipment/factory  tng 
Other 

Weapon  system  cost 

Flyaway  cost 

2  04  Data 

2  09  Peculiar  support  equip  ASIDE 

2.06  Training  services  &  equipment 

2.08  Operational  site  activity 

Procurement  costs 

Weapon  system  costs  plus 

Initial  spares 

Other 

Procurement  costs 

Weapon  system  costs  plus 

2  07  Initial  spares 

4.0  Fielding  (procurement) 

Program  acquisition  costs 

Procurement  costs  plus 

RDTE 

Facility  construction 

Program  acquisition  costs 

Procurement  costs  plus 

1.0  Development 

3.0  Military  construction 

Life  cycle  costs 

Program  acquisition  costs 

Life  cycle  costs 

Program  acquisition  costs 

Operations  &  support 

5.0  Sustainment  +  4.0  Fielding  (OMA) 

Common  support  equipment 

2  092  Common  equip  (ASlOE)  +  2  093  OSE 

Disposal 

N/A 

Total  lifecycle  costs 
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b.  Most  of  the  data  were  provided  in  the  form  of  BCEs  from  the  program 
managers  (PMs)  through  CE  AC.  Other  sources  included  the  Assistant  Secretary  of 
the  Army  for  Research,  Development,  and  Acquisition  (ASARDA),  Army  Materiel 
Command  (AMC),  and  ODCSOPS.  Cost-quantity  curves  and  FUCs  were  obtained 
from  the  PMs  or  derived  from  production  data  by  CAA,  when  applicable. 

c.  BCEs  are  not  provided  in  consistent  formats.  Hence,  there  were  significant 
gaps  in  the  LCCM  data  for  some  systems.  In  some  instances,  portions  of  the  data 
provided  in  a  BCE  were  extrapolated  by  CAA  to  provide  complete  life  cycle  costs  (see 
paragraph  7-3b). 

7-3.  VAA  COSTING  ASSUMPTIONS.  The  following  assumptions  were  made  for 
all  systems  in  the  VAA  optimization  input  and  life  cycle  costing  processes. 

a.  VAA  Optimization  Input 

(1)  The  following  production  costs  vary  directly  with  production  quantity: 
recurring  engineering,  tooling,  quality  control,  initial  spares,  and  training  services 
and  equipment. 

(2)  The  following  production  costs  do  not  vary  directly  with  production 
quantity;  nonrecurring,  engineering  changes,  data  syscem  evaluation,  operation  of 
site  activity,  other  production  excluding  peculiar  associated  items  of  equipment 
(ASIOE),  and  organization  support  equipment  (OSE). 

b.  Life  Cycle  Costing 

(1)  RDTE  and  MCA  costs  remain  constant  with  the  baseline  and  do  not  change 
with  varying  production  schemes. 

(2)  Analogous  system  cost  data  can  be  applied  to  future  system  cost  data  when 
future  system  cost  data  are  not  available. 

(3)  Proportional  unit  costs,  by  P-92  cost  data,  from  BCE/ACP  baseline  costs  can 
be  applied  to  new  production  quantities. 

(4)  Fielding  will  take  place  2  years  after  production, 

(5)  Operation  and  sustainment  costs  are  based  on  cumulative  fielding  of 
weapons  systems. 

(6)  O&S  cost  buildup  begins  when  the  first  system  produced  is  fielded. 

(7)  When  transportation  costs  do  not  distinguish  between  first  destination 
transportation  (FDT)  costs  and  second  destination  transportation  (SDT)  costs,  FDT 
comprise  1/3  of  the  transportation  costs  and  SDT  comprise  2/3  of  the  total 
transportation  costs. 
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7-4.  VAA  COSTING  LIMITATIONS.  The  following  limitations  are  applicable  to 
the  optimization  input  and  life  cycle  costing  processes. 

a.  VAA  Optimization  Input 

(1)  Economies  of  scale  are  reflected  by  existing  or  derived  composite  cost- 
quantity  curves. 

(2)  BCE/ACP  data  are  used  for  VAA  base  case  unit  production  costs  and 
quantities. 

(3)  FUCs  and  cost-quantity  curves  for  weapon  systems  are  applied  to  the  total 
system  rather  than  a  summation  of  costs  resulting  in  the  application  of  system 
component  cost-quantity  curves  and  FUCs. 

(4)  Average  unit  procurement  costs  are  used  when  cost-quantity  curves  are  not 
available  (as  was  the  case  for  some  pre-Milestone  2  systems)  or  appropriate  (as  was 
the  case  for  some  combat  service  support  (CSS)  mission  area  systems). 

b.  Life  Cycle  Costing 

(1)  Production,  fielding,  and  sustainment  P-92  subelements  for  optimized 
quantities  are  calculated  based  on  the  same  proportions  as  the  baseline. 

(2)  Data  for  previous  years  and  outyears  (years  beyond  2008)  are  grouped. 

(3)  Costs  for  the  outyears  are  based  on  the  difference  between  the  costs  prior  to 
2009  and  the  total  costs  provided. 

(4)  Categories  of  Procurement,  Army  (PA)  appropriations  are  not  provided  (i.e., 
missiles,  aircraft). 

7-5.  VAA  OPTIMIZATION  INPUT 

a.  The  input  cost  data  to  the  VAA  optimization  model  are  provided  in  constant  FY 
92  dollars.  For  systems  with  cost-quantity  curves,  the  input  data  include  previously 
produced  quantities,  first  unit  costs  (FUCs),  and  the  cost-quantity  slope  coefficient  for 
recurring  manufacturing  costs,  annual  variable  unit  costs,  and  annual  fixed  costs. 
Recurring  manufacturing  cost  is  a  key  production  cost  driver.  A  sample  of  FUCs  and 
cost-quantity  curves  used  is  shown  in  Table  7-2.  This  data  reflects  recurring 
manufacturing  costs  except  for  five  ASM  systems  (Block  III,  future  infantry  fighting 
vehicle  (FIFV),  combat  mobility  vehicle  (CMV),  future  armored  resupply  vehicle- 
artillery  (FARV-A),  and  Advanced  Field  Artillery  System  (AFAS))  which  reflect  total 
production  costs  for  a  revamped  ASM  program.  Other  exceptions  are  footnoted  in  the 
table. 
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Table  7-2.  First  Unit  Costs  and  Cost-quantity  Curves 


System 

First  unit  cost 
(constant 

FY  92  $M) 

Cost-qty 

curve 

(percent) 

Prior  qtys  (prior 
to  FY  94) 

AMS-Ha 

$.0657 

87.7 

AAWS-M 

.1796 

87.7 

ADATS 

11.38 

92.5 

11 

ATAS 

.068 

93.3 

AVENGER 

.90 

97.1 

629 

SADARM 

(MLRS) 

.4363 

86.5 

VEMASIDb 

.0657 

94.6 

540 

AGSb 

2.7 

95.3 

ATACMS  II 

2.45 

89.5 

GBS 

4.92 

88.9 

NLOS-AT 

2.88 

94.1 

LOSAT 

11.27 

91.3 

STINGRAY 

1.28 

87.6 

AFAS 

15.73 

90.0 

BLOCK  III 

34.4 

90.0 

CMV 

15.47 

90.0 

FARV-A 

8.89 

90.0 

FIFV 

35.7 

90.0 

RAH-66 

37.79 

87.5 

aReflects  total  recurring  costs. 
bReflects  total  production  costs. 


b.  Table  7-3  provides  the  P-92  cost  categories  that  are  considered  fixed  costs  and 
those  considered  variable  costs.  Thus,  the  annual  procurement  costs  for  optimized 
quantities  were  calculated  by  summing  the  recurring  manufacturing  costs  (computed 
from  the  first  unit  cost  and  cost-quantity  slope),  the  unit  variable  costs  (multiplied  by 
the  new  quantity),  and  the  fixed  costs  by  year. 
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Table  7-3.  Composition  of  Procurement  (-)  Costs 


PROCUREMENT  COSTS  (-)  - 


RECURRING  MANUFACTURING  COSTS  +  VARIABLE  COSTS  +  FIXED  COSTS 


Recurring  Costs 


2.021  Recurring  Manufacturing  Costs  based  on 
Learning  Costs. 


Variable  Costs 


=  2.022  Recurring  Engineering  +  2.023  Tooling 
+  2.024  Quality  Control  +  2.06  Training  Services 
+  2.07  Initial  Spares. 


Fixed  Costs 


=  2.01  Nonrecurring  Costs  +  2.03  Engineering  Changes 
+  2.04  Data  +  2.05  System  Evaluation 
+2.08  Operate  Site  Activity 

+  2.09  Other  Production  (excluding  2.092  Peculiar  ASIOE  and 
2.093  Organizational  Support  Equipment). 


c.  Procurement  (-)  costs  for  VAA  Phase  11  drills  exclude  associated  support  items 
of  equipment  (ASIOE)  and  procurement  funded  fielding  to  be  consistent  with  what  is 
included  in  the  MDEPs  identified  with  weapon  systems.  ASIOE  and  procurement 
funded  fielding  cannot  always  be  directly  correlated  with  a  specific  weapon  system 
MDEP. 

d.  A  software  package  developed  for  the  Air  Force  called  LCURVE  (LCURVE, 
Version  1.3,  Telecote  Research,  Inc.,  15  Feb  91)  was  used  to  derive  cost-quantity 
curves  for  most  systems.  A  detailed  explanation  of  the  derivation  of  cost-quantity 
curves  is  provided  in  Appendix  F  of  the  VAA  Phase  I  Study  Report.  FUCs  and  cost- 
quantity  curves  that  were  derived  were  rejected  if  the  coefficient  of  determination 
(R2)  was  less  than  85  percent  or  if  resulting  curves  were  lower  than  85  percent  (This 
was  based  on  discussions  with  CEAC  and  PMs).  In  most  cases,  cost-quantity  curves 
were  applied  to  recurring  manufacturing  costs  only,  with  the  remaining  costs  treated 
as  fixed  production  costs  by  year  or  unit  variable  costs.  In  other  instances,  detailed 
production  costs  were  not  available,  and  total  recurring  manufacturing  costs  or  total 
production  costs  were  used  to  derive  a  cost-quantity  curve. 

e.  Average  unit  procurement  costs  by  year  were  derived  froni  the  baseline  (i.e., 
BCE)  for  weapon  systems  for  which  cost-quantity  curves  were  not  used.  Procurement 
costs  for  those  systems  without  cost-quantity  curves  were  computed  by  multiplying 
average  unit  production  costs  by  optimized  quantities. 
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f.  Procurement  costs  for  VAA  weapon  systems  were  modified  to  exclude  ASIOE 
and  OSE  and  then  added  to  RDTE  costs  to  compute  a  program  acquisition  (-)  cost 
consistent  with  the  MDEP  appropriation  configuration  of  systems  in  the  LRRDAP 
data  structure.  Procurement  funded  fielding  costs  and  MCA  costs  were  also  excluded 
from  the  program  acquisition  (-)  cost  calculation. 

g.  An  algorithm  to  dynamically  account  for  cost-quantity  curves  was  incorporated 
in  the  optimization  model.  An  explanation  as  to  how  the  optimization  model 
dynamically  computes  costs  is  provided  in  Chapter  8. 

h.  Annual  minimum  sustainment  rates  (MSRs)  and  maximum  production  rates 
(MPRs)  were  obtained  from  procurement  forms’  (P-form)  data.  MSRs/IVIPRs  for 
systems  without  P-form  data  were  provided  by  ASARDA  and  from  data  provided  by 
PMsto  AMC. 

(1)  MSR  is  defined  as  a  level  of  production  which  will  keep  a  production  line 
open  while  maintaining  a  responsive  vendor  and  supplier  base  (i.e.,  warm  production 
base).  The  assumption  inherent  in  the  MSR  is  that  the  plant  must  operate  one  shift, 

8  hours  per  shift,  5  days  a  week.  Conversely,  the  MPR  is  defined  as  a  level  of 
production  which  maximizes  the  capacity  of  the  existing  tooling  or  facilities  without 
requiring  an  additional  investment  to  increase  the  production  capacity.  This 
definition  assumes  three  shifts  of  8  hours  per  shift,  7  days  a  week. 

(2)  MSRs  and  MPRs  for  some  systems  were  adjusted  to  reflect  a  ramp-up  to  full- 
scale  production,  based  upon  baseline  procurement  quantities. 

(3)  It  should  be  reiterated  that,  for  the  VAA  Phase  II  drills,  the  optimization 
model  was  constrained  to  procure  the  total  force  structure  requirement  as  defined  by 
the  sponsor.  If  the  force  structure  requirement  was  not  met,  then  the  system  was  not 
procured.  Force  structure  requirements  are  shown  in  Chapter  4.  Production 
constraints  were  not  available  for  a  few  developmental  systems  and  were  set  based  on 
analogous  systems  and  planned  production  schedules.  Table  7-4  provides  the  MSRs 
and  MPRs  used  by  each  system. 
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Table  7-4.  Annual  MSRs  and  MPRs  Used  in  VAA 


Optimization  ModeKpage  1  of  2  pages) 


System 

Annual  iVISR 
(each) 

Annual  MPR 
(each) 

AAWS  M 

996 

20,000 

ADATS 

12 

72 

ADV  QUICKFIX 

5 

100 

ARV 

50 

100 

ATAS 

36 

480 

AVENGER 

24 

280 

FMTV  5-TON 

250 

2,500 

GBCS-L/H 

3 

50 

HAB 

20 

100 

HET 

150 

500 

JSTARS 

5 

15 

M1A2 

120 

1,440 

SADARM  IMLRS) 

1,200 

7,500 

PLS 

280 

3,000 

T0W2B 

8,000 

36,000 

1 

VEMASID 

30 

150 

GROUND  VOLCANO 

21,000 

135,000 

AIR  VOLCANO 

21,000 

135,000 

WAM 

1,800 

4,667 

155  ER 

4,176 

35,280 

155  LT  WT 

24 

192 

155  TGP 

348 

2,940 

ADV  FIRE  FINDER 

3 

10 
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Table  7-4.  Annual  MSRs  and  MPRs  Used  in  VAA 


Optimization  Model  (page  2  of  2  pages) 


System 

Annual  MSR 
(each) 

Annual  MPR 
(each) 

AGS 

24 

360 

AH-64  LBa 

48 

144 

AMS-Ha 

2,000 

25,000 

ATACMS  II 

240 

864 

GBS 

12 

48 

LOSATa 

34 

114 

MLRS  TGW 

1,200 

4,000 

NLOS-AT 

36 

422 

RAH-66a 

72 

168 

STINGRAY 

15 

60 

ADVSEMA 

3 

25 

AFASa 

72 

228 

BLOCK  Ilia 

240 

636 

CMVa 

48 

76 

FARV-Aa 

72 

228 

FIFVa 

72 

228 

RAH-66  LBa 

48 

144 

FMTV  2  1/2-TON 

515 

3,250 

aNOTE:  annual  ramp-up  constraints  were  used  for  these 
systems. 
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7-6.  ANNUAL  TOA  BUDGET  CONSTRAINT  COMPUTATION 

a.  A  series  of  optimization  runs  was  performed  to  support  the  LRRDAP  budget 
drill  process.  An  estimated  budget  constraint  was  produced  to  allocate  the  proportion 
of  a  given  RDA  TOA  to  VAA  systems,  as  illustrated  in  Figure  7-3. 


PA&E 


VAA  Proportion 
of  RDA  S 

from  LRRDAP  File 


VAA  TOA  S  constraints 


Figure  7-3.  Annual  TOA  Budget  Constraint  Computation 

b.  TOA  was  provided  by  standard  study  number  (SSN)  for  procurement  dollars 
and  program  element  (PE)  for  RDTE  dollars  in  the  LRRDAP  data  base.  The  MDEP 
codes  were  manually  examined  along  with  the  appropriate  PEs  and  SSNs  and 
assigned  to  the  respective  VAA  weapon  category.  The  current  dollars  were  then 
deflated  to  constant  FY  92  dollars.  A  proportion  of  VAA  system  dollars  to  total  TOA, 
excluding  dollars  designated  as  "fixed"  by  PA&E  (technology  base,  base  operations 
support,  test  and  evaluation,  and  training  devices)  was  used  to  estimate  the  TOA 
constraint  for  the  VAA  portion  of  the  RDA  account. 

(1)  The  first  step  was  to  sum  the  RDA  (including  RDTE,  Procurement,  Army 
(PA),  Other  Procurement,  Army  (OPA),  missiles,  wheeled  and  tracked  vehicles 
(WTCV),  aircraft,  etc.)  dollars  in  the  pa  rticular  version  of  LRRDAP  being  addressed 
for  each  year  of  the  POM  and  EPP  for  VAA  systems  (FY  94  to  FY  08).  There  was  a 
very  small  margin  of  error  in  this  calculation,  since  some  MDEPs  (i.e.,  trucks,  mines, 
etc.)  are  attributed  to  multiple  systems. 

(2)  The  second  step  was  to  calculate  the  total  RDA  dollars  for  all  MDEPs 
reflected  in  the  LRRDAP  for  both  the  POM  and  EPP  years. 
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(3)  Fixed  costs  were  subtracted  from  the  total  RDA  ($),  and  a  ratio  was  derived 
based  on  the  proportion  of  dollars  assigned  to  VAA  systems  dollars  to  the  total  RDA 
($)  minus  the  fixed  portion.  This  ratio  was  applied  to  a  given  annual  TOA  constraint 
(e.g.,  $12.7  billion  per  year).  This  fixed  annual  TOA  constraint  in  the  EPP  was 
inflated  based  on  the  latest  OSD  inflation  guidance  of  3.2  percent  per  annum. 

7-7.  LIFE  CYCLE  COSTING 

a.  Introduction.  Life  cycle  costs  were  based  on  the  BCE  P-92  cost  code  structure. 
Table  E-1  depicts  the  P-92  codes  and  corresponding  appropriations  used  in  the  life 
cycle  costing.  The  LCCM  used  in  VAA  is  a  Quattro  Pro  spreadsheet  model  that 
provides  the  analyst  a  quick  estimate  of  the  impacts  that  changes  in  procurement 
quantities  have  on  the  various  P-92  cost  elements,  appropriations,  and  categories  of 
weapon  system  costs.  The  upper  portion  of  the  spreadsheet  (LCCMl)  contains  the  P- 
92  cost  code  data  obtained  from  BCE/ACP  data;  and  the  lower  part  (LCCM2)  provides 
proportional  estimates  of  the  same  P-92  cost  code  data  for  new  procurement 
quantities,  based  on  the  original  BCE/ACP  data  relationship.  Appendix  E  provides 
an  example  of  LCCMl  and  LCCM2. 

b.  Life  Cycle  Cost  Model.  The  upper  part  of  the  LCCM  (LCCMl)  contains  the 
BCE/ACP  data  and  drives  the  lower  part  (LCCM2),  which  provides  the  life  cycle  cost 
estimates  for  the  optimized  quantities.  The  entire  LCCM  is  driven  by  two  pieces  of 
input  data:  annual  average  unit  production  cost  and  the  associated  optimized 
quantities.  This  data  is  obtained  from  the  optimization  model  output.  As  discussed 
earlier  in  the  the  cost-quantity  methodology,  the  optimization  model  considers  the 
effects  of  "learning”  and  "economies  of  scale."  Some  minor  adjustment  to  the  output 
costs  may  be  required  in  applications  where  ASIOE  and  OSE  are  removed,  as  was  the 
case  in  the  VAA  support  for  the  LRRDAP  drills  (production  costs  include  ASIOE  and 
OSE  costs). 

(1)  RDI'E  and  MCA  Cost.  The  1.0  RDTE  and  3.0  MCA  costs  are  considered 
fixed  costs  regardless  of  production  quantities.  Thus,  the  costs  for  LCCM2  are 
identical  to  the  LCCMl  baseline.  Generally,  MCA  costs  were  absent  from  the 
BCE/ACP. 

(2)  Production  Costs.  The  2.0  production  costs  and  quantities  are  obtained 
from  the  optimization  results.  The  equation  that  represents  the  LCCM2  2.0 
production  cost  stubs  is  as  follows: 


LCCM2  2.0  Optimiz>’d  Production  Qty 
LCCMl  2  0  Rasclinr  Production  Qty 


*  ( LCCMl  2  0  P-92  cost  code) 


When  there  are  costs  associated  with  2.0  production  cost  categories  and  no  quantities 
in  LCCMl,  these  costs  are  carried  over  to  LCCM2  directly  without  proportional 
adjustment.  These  are  startup  costs  prior  to  production  and  assumed  to  be 
unchanged  by  the  variation  in  the  production  scheme. 
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(3)  Fielding  Costs.  Fielding  quantities  are  taken  from  the  production 
schedule  based  on  the  annual  optimized  quantities  and  shifted  2  years  in  the  future. 
The  assumption  is  that  fielding  will  occur  2  years  after  production.  As  with 
production  cost  stubs,  a  proportional  calculation  for  the  4.0  fielding  cost  stubs  was 
made  based  on  the  LCCMl  baseline: 


LCCM2  4.0  Mew  Fielding  Qty 
LCCMl  4.0  Baseline  Fielding  Qty 


*  {LCCMl  A. H  P —  ^2  cost  code) 


(4)  Sustainment  Costs.  Sustainment  quantities  are  calculated  similar  to  the 
cumulative  fielding  quantities  with  sustainment  costs  being  incurred  in  the  first  year 
of  fielding.  Again,  costs  are  proportionally  derived  from  the  LCCMl  baseline: 


LCCM2  5.0  New  Sustainment  Qty 
LCCMl  5.0  Baseline  Sustainment  Qty 


*  {LCCMl  5.0  P- 92  cost  code) 


c.  Inflation.  Procurement  and  RDTE  costs  for  weapon  systems  obtained  in 
constant  dollars  were  inflated  using  composite  indices  from  AMC  guidance.  The 
composite  index,  used  when  including  inflation  in  TOA,  not  only  incorporates  the 
mathematical  principle  of  compounding,  but  also  takes  into  consideration  outlay 
rates  and  the  impact  of  inflation  over  the  entire  spending  pattern.  Compound  indices 
were  applied  to  OMA,  MPA,  and  MCA  funded  life  cycle  costs  when  required. 

d.  Major  Weapon  Systems  Cost  Categories.  The  following  five  major  weapon 
system  cost  categories  were  computed  from  the  baseline:  flyaway,  weapon  system, 
procurement,  program  acquisition,  and  total  life  cycle  costs.  The  composition  of  these 
major  weapon  system  cost  categories  is  provided  in  Table  7-4. 

e.  Appropriations.  The  RDTE,  PA,  OMA,  Army  Stock  Fund  (ASF),  and  MPA 
appropriations  are  calculated  by  summing  the  P-92  cost  codes  by  appropriation. 

f.  Data  Quality.  Each  spreadsheet  for  a  weapon  system  was  categorized  as  to  the 
completeness  of  the  data  as  follows: 

(1)  Category  A  -  comprehensive  data  obtained  from  BCE. 

(2)  Category  B  -  data  from  BCE,  but  either  some  extrapolation  by  CAA  or 
significant  gaps  in  the  data. 

(3)  Category  C  -  data  largely  limited  to  rolled  RDTE  and  production  costs.  This 
category  consists  of  mostly  very  developmental  systems  that  are  not  at  the  stage 
requiring  a  BCE. 
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g.  Validation.  Proportional  factors  from  the  February  1990  ASM  BCE  were 
compared  to  the  cost  codes  provided  in  the  later  April  1990  ASM  BCE  for  reduced 
quantities.  Comparisons  were  made  for  the  Block  III  tank,  FIFV,  CMV,  FARV-A, 
and  AFAS.  Proportions  remained  relatively  constant,  under  5  percent  deviation  for 
most  P-92  cost  codes.  Table  7-5  shows  the  deviations  of  the  estimated  costs  using  the 
computed  proportional  factors  compared  to  the  updated  ASM  BCE. 


Table  7-5.  Block  III  Comparison 


P-92 

cost 


codes 

BLK  III  Comparison 

FY  01 

FY  02 

FY  03 

FY  04 

2.01 

Nonrecur 

0.3% 

0.3? 

0.3? 

0.4? 

2.02 

Recur 

2.4? 

2.9? 

2.7? 

3.5? 

2.021 

Manufacturing 

4.0? 

4.8? 

4.3? 

5.5? 

2.022 

Recurring  Engineering 

0.9? 

1.1? 

1.0? 

1 .2? 

2.023 

Tool  ing 

0.4? 

0.4? 

0.4? 

0.5? 

2.024 

Quality  Control 

0.2? 

0.3? 

0.3? 

0.3? 

2.03 

Engineering  Changes 

0.2? 

0.2? 

0.2? 

0.4? 

2.04 

Data 

0.7? 

0.8? 

0.7? 

0.9? 

2.05 

Sys  Test  &  Eval 

0.1? 

0.1? 

0. 1? 

0.2? 

2.06 

Training  Serv  &  Equip 

0.0? 

0.2? 

0.2? 

0. 1? 

2.07 

Initial  Spares 

0.2? 

0.1? 

0.2? 

0.5? 

2.09 

Other  Proc  Funded  Prod 

0,9? 

1.2? 

1.0? 

1.3% 

4.02 

Training,  Serv  ^  Equip 

2.6? 

1.3? 

2.0? 

2.1? 

4.03 

Transportation 

1.7? 

1.5? 

1.5? 

1 .5? 

4.04 

Initial  Repair  Parts 

4.3? 

2.9? 

3.5? 

3.6? 

5.01 

Replen ishment 

3.9? 

7.1? 

7.0? 

6.7? 

5.02 

Petrol , Oil , Lubricants 

0.0? 

0.0? 

0.0? 

0.0? 

5.03 

Ammo/Msl 

2.8? 

8.5? 

6 . 6? 

5.4? 

5.04 

Depot  Maintenance 

1.8? 

1.0? 

0.8? 

0.8? 

5.041 

Depot  Maint  Civ.  Labor 

0.7? 

0.4? 

0.3? 

0.3? 

5.05 

Field  Maint  Civ.  Labor 

0.0? 

0.0? 

0.0? 

0.0? 

5.06 

Transportation 

0.0? 

0.1? 

0.1? 

0. 1? 

5.07 

Repl  Training 

2.7? 

1.3? 

0.8? 

0.6? 

5.08 

MPA 

0.3? 

1  .2? 

2.4? 

3.0? 

5.09 

Sys/Proj  Mgmt(Civ) 

0.9? 

0.3? 

0.2? 

0.2? 

5.10 

Modifications /Kits 

1.3? 

0.2? 

0.4? 

0.3? 

5.11 

Other  Sustainment 

0.3? 

0.  1? 

0.  1? 

0.0? 

5.111 

Other  OMA  Funded 

0.3? 

0.2? 

0.2? 

0.2? 

5.112 

Other  PA  Funded 

0.0? 

0.0? 

0.1? 

0.1? 

NO'l'E:  Block  III  quantities  were  3,349  in  Feb  91  BCE  and  1,946  in  April  1991 
BCE.  Annual  production  rates  compared  above  are  500  per  year  versus  300  per 
year. 
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The  percent  differences  between  cost  code  proportion  of  total  production,  fielding,  and 
sustainment  costs  are  shown  in  Table  7-5  comparing  the  Block  HI  February  1991 
BCE  to  the  Block  EQ  April  1991  BCE. 

h.  LCCM-Shift.  This  paragraph  summarizes  the  process  that  was  used  to 
develop  a  condensed  life  cycle  cost  model  that  automatically  adjusts  cost  with  shifts 
in  production  schedules.  Currently,  the  condensed  Life  Cycle  Cost  Model  (LCCM- 
Shift)  is  planned  for  use  in  VAA  Phase  III. 

(1)  One  of  the  advantages  of  using  this  revised  spreadsheet  version  is  to  provide 
the  flexibility  of  having  a  quick  turnaround  of  pertinent  cost  data  with  the  capability 
of  revising  planned  production  schedules.  This  spreadsheet  is  unlike  the  original 
LCCM,  which  is  extracted  from  a  greater  level  of  detail  from  a  weapon  system's  BCE. 
Dollars  are  adjusted  for  different  years  with  scheduled  production  shifts  from  the 
beginning  of  the  POM  until  the  end  of  the  EPP  (FY  94  -  FY  08),  for  any  combination 
of  years. 

(2)  The  spreadsheet  model  uses  several  macros  within  a  Quattro  Pro 
spreadsheet.  The  data  is  based  on  the  most  appropriate  data  elements  from  the  BCEs 
of  each  of  the  respective  systems  (see  format  below).  The  only  user  inputs  within  the 
macro  are  the  start  years  where  there  are  both  quantities  and  production  dollars  for 
the  baseline  and  revised  entries.  Again,  new  quantities  and  average  unit  costs  must 
be  determined  and  entered. 

(3)  The  production,  fielding,  and  sustainment  costs  are  based  on  the  ratio  of 
optimization  production  dollars  to  corresponding  baseline  production  dollars  (first 
year  of  baseline  versus  first  year  of  optimization,  etc.).  Output  is  provided  in  the 
following  formats: 

(a)  Baseline  (constant  dollars), 

(b)  Baseline  (current  dollars), 

(c)  Revised  entries  (constant  dollars), 

(d)  Revised  entries  (current  dollars),  and 

(e)  Revised  Big  5  (current  dollars). 

(4)  The  dollars  for  each  stub  are  proportionally  inflated  or  deflated,  depending 
on  when  the  new  optimized  production  plan  is  scheduled,  either  before  or  after  the 
baseline.  The  deflation/inflation  factors  are  from  the  updated  AMC  Inflation 
Guidance  memo  dated  January  1992. 

(5)  An  illustration  of  the  baseline  template  is  shown  in  Table  7-6.  It  is 
approximately  one-third  of  the  size  of  the  original  LCCM.  System  costs  in  the  form  of 
flyaway,  weapon  system,  procurement,  program  acquisition,  and  life  cycle  costs  have 
been  derived  from  historical  BCE  data  for  the  category  A  systems  (see  paragraph  7-7f 
for  description  of  category  A). 
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Table  7-6.  LCCM  Shift 


Revised  LCCM  template 

P-92  cost 
category 

Cost  element 

1 .0 

Development 

2.0 

Production  (total) 

QTY:  (primary) 

QTY:  (secondary) 

2.01 

Nonrecur 

2.02 

Recur 

2.021 

Manufacturing 

2.022 

Engineering 

2.023 

Tooling 

2.024 

Quality  control 

2.03 

Engineering  changes 

2.04 

Data 

2.05 

Sys  test  4  eval 

2.06 

Training  serv  4  equip 

2.07 

Initial  spares 

2.08 

Operate/site  activity 

2.09 

Other  proc  funded  prod 

2.091 

System  proj  mgmt 

2.092 

Peculiar  spt  equip 

2.092 

Common  spt  equip 

2.093 

Org  support  equip 

2.094 

Other  prod  cost 

2.10 

System  project  mgmt 

3.0 

Military  construction 

4.0 

Fielding  (total) 

4.0 

Fielding  (OMA) 

4.0 

Fielding  (PA) 

5.0 

Sustainment  (total) 

5.0 

Sustainment  (OMA) 

5.0 

Sustainment  (P'A) 

5.0 

Sustainment  (MPA) 

5.0 

Sustainment  (ASF) 

7-8.  METAPHOR  USE 

a.  The  LCCM  spreadsheets  (baseline  data)  were  exported  into  the  METAPHOR 
computer  at  ODCSOPS  to  provide  access  for  action  ofilcers  at  the  Pentagon.  The 
formulas  to  generate  the  second  part  of  the  LCCM  were  put  in  a  capsule  to  generate 
the  life  cycle  cost  for  the  optimized  quantities.  C  AA  can  transfer  the  optimized 
quantities  from  the  VAA  optimization  runs  to  the  METAPHOR  data  base  via 
electronic  mail.  The  LCCM  (part  2)  on  the  METAPHOR  computer  provides  the 
capability  for  the  action  officer  at  the  Pentagon  to  estimate  impacts  that  VAA 
optimized  quantities  have  on  P-92  cost  elements,  appropriations,  and  weapon  system 
cost  categories.  Users  can  also  perform  "what  if  drills  using  the  METAPHOR 
computer. 
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b.  Capsules  were  created  to  produce  standard  reports  showing  the  five  major  cost 
categories,  appropriation  categories,  and  weapon  system  categories.  The  goal  of  this 
approach  is  to  allow  for  routine  queries  and  reports  of  selected  data  in  a  variety  of 
formats  such  as  by  system  or  mission  area.  For  example,  a  listing  of  all  recurring  cost 
production  dollars  for  each  system  for  the  total  POM  period  may  be  of  interest. 
Similarly,  a  sum  of  the  total  life  cycle  sustainment  costs  for  a  selected  mission  area 
may  be  required.  Additional  reports  and  queries  can  be  generated  as  required. 

7-9.  ASIOE  PERSPECTIVE.  Major  item  system  maps  (MISMs)  for  approximately 
20  VAA  systems  were  obtained  from  US  AMC  Systems  Integration  and  Management 
Activity  (USAMC  SIMA).  These  maps  provided  the  ASIOE  based  on  the  original 
basis  of  issue  plan  (BOIP)  which  usually  differed  from  the  PM's  ASIOE  costed  in  the 
BCE.  Costs  by  line  item  number  (LIN)  were  obtained  from  The  Army  Force  Cost 
System  (TAFCS)  data  base  and  can  be  multiplied  by  the  per-system  ratios  provided  in 
the  MISM.  In  some  instances,  ratios  were  imputed  to  a  per-system  factor  from  a  per- 
unit  factor. 

7-10.  SUMMARY.  VAA  costs  were  generated  to  provide  a  complete  life  cycle 
picture  considering  cost-quantity  relationships  and  the  requirement  to  meet  the 
quick  reaction  demands  of  the  POM  building  process.  The  VAA  costing  methodology 
allows  for  the  capability  to  produce  estimates  of  procurement  costs,  RDA  costs,  VAA 
system  budget  constraints,  P-92  cost  categories,  weapon  system  cost  categories,  and 
appropriation  categories  quickly  and  within  an  acceptable  degree  of  accuracy.  The 
cost  module  is  flexible  enough  to  allow  for  changes  in  the  cost  perspective  view 
desired  by  the  decisionmaker.  Further  refinements  are  expected  with  the  continued 
use  of  the  VAA  process. 
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CHAPTERS 

COST  EFFECTIVENESS  ANALYSIS 


8-1.  COST  EFFECTIVENESS  ANALYSIS  APPROACH 

a.  Introduction.  The  final  determination  of  a  recommended  acquisition  strategy 
requires  the  consideration  of  many  factors.  These  factors  include  the  effectiveness  of 
the  systems  under  consideration,  how  much  the  systems  cost,  the  force  structure 
requirement  for  the  systems  to  fill  the  force,  and  the  capability  of  industry  to  produce 
the  systems.  Difficulty  has  long  existed  in  keeping  track  of  these  diverse  factors  in 
the  process  of  determining  what  to  procure,  in  what  quantities,  and  in  what 
timeframes  to  achieve  the  most  cost  effective  force  possible. 

b.  Optimization  in  Value  Added  Analysis  Phase  I  Study.  In  the  VAA  Phase  I 
Study,  the  decision  was  made  to  use  mathematical  programing  to  perform  this  cost 
effectiveness  analysis  and  to  produce  a  recommended  acquisition  strategy.  A 
prototype  linear  programing  (LP)  formulation  was  devised  for  that  purpose.  This 
formulation  was  represented  as  follows: 

Maximize:  Force  effectiveness 

Subject  to;  Budget  ceiling 

Force  structure  requirements 
Production  limitations 

Although  the  model  was  usable  and  valuable  insights  were  realized  through  its  use, 
several  shortcomings  of  the  prototype  formulation  were  identified  during  VAA  Phase 
I.  First,  if  insufficient  funds  were  available  to  procure  the  minimum  required 
quantities  of  each  system  under  consideration  in  each  year,  the  problem  would  be 
infeasible.  The  model  had  no  capability  to  identify  systems  that  should  be  "cut"  from 
the  program.  Since  more  programs  are  available  for  procurement  than  there  are 
funds  available  to  pay  for  them,  it  was  clear  that  some  method  of  identifying  systems 
that  should  be  left  unfunded  was  needed.  In  VAA  Phase  I,  identification  of  cuts  was 
performed  iteratively  based  on  the  cost-benefit  ratios  of  the  systems.  This  process 
was  slow  and  tedious  and  did  not  guarantee  optimal  results.  Second,  all  costs, 
including  fixed  manufacturing  costs  and  RDTE  costs,  which  were  not  related  to 
quantity  had  to  be  included  in  an  annual  average  unit  cost  for  each  system.  Third, 
the  model  could  not  handle  the  so-called  "learning  curve"  costs.  These  costs  are 
discussed  in  Chapter  7,  and  significantly  affect  the  per-unit  costs  of  many  systems. 
Thus,  the  variable  manufacturing  costs  were  often  inaccurate. 

8-2.  ANALYTICAL  MODELS  AND  TOOLS 

a.  Enhanced  Optimization.  In  order  to  overcome  the  above-stated 
shortcomings,  the  VALOR  Model  was  developed.  VALOR  is  a  mixed  integer  program 
(MIP).  Since  the  decision  was  made  to  use  a  MIP  formulation,  the  shortcomings 
described  above  could  be  handled  in  a  relatively  standard  manner.  The  details  of  the 
VALOR  formulation  and  implementation  can  be  found  in  the  Technical  Paper,  CAA- 
TP-92-1.  A  less  detailed  discussion  of  the  VALOR  Model  is  given  below. 
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b.  Objective  Function.  The  objective  of  VALOR,  and  indeed  in  VAA  in  general, 
is  to  suggest  a  mix  of  systems  for  procurement  that  will  be  as  effective  as  possible  in 
combat,  subject  to  constraints  on  budget,  force  structure,  and  production  capabilities. 
The  effectiveness  of  the  various  candidate  systems  is  evaluated  and  quantified  in  the 
Explicit  and  Implicit  Effectiveness  Modules  of  the  VAA  methodology,  and  these 
various  measures  are  integrated  using  TOPSIS  in  the  Effectiveness  Integration 
Module  to  produce  the  Value  Added  Coefficient.  The  result  is  a  single  measure  of  a 
system's  contribution  to  the  effectiveness  of  the  overall  force  for  each  scenario  and  for 
each  year  the  system  will  be  in  the  force.  These  measures  are  then  used  to  form  the 
objective  function  coefficients. 

(1)  Forming  the  Objective  Coefficients.  This  so-called  Value  Added 
Coefficient  is  related  to  the  presence  or  absence  of  the  system  on  the  battlefield.  The 
combat  results  are  derived  using  the  RSM  experimental  design  and  relate  to  whether 
or  not  the  entire  program  is  in  the  force.  Whole  programs  are  evaluated  in  the 
Implicit  Effectiveness  Module.  Thus,  the  need  existed  to  convert  this  measure  of 
"program"  effectiveness  to  a  "per- system"  measure.  This  conversion  was 
accomplished  by  dividing  each  Value  Added  Coefficient  by  the  appropriate  density  of 
the  system  played  in  CORBAN  giving  a  new  value  which,  when  multiplied  by  the 
quantity  of  systems  to  be  procured,  would  give  the  the  effectiveness  of  the  entire 
quantity.  Note  that  this  procedure  would  not  be  appropriate  unless  the  quantity  of 
each  system  procured  was  related  to  the  CORBAN  densities.  In  VAA,  however, 
procurement  quantities  are  constrained  to  be  the  amount  necessary  to  fill  some  force 
structure  requirement.  As  such,  the  unreasonable  situation  where,  for  instance,  only 
one  item  is  procured  is  precluded. 

(2)  Coefficients  for  Intervening  Years.  In  VAA  Phase  II,  only  the  years 
1996,  2001,  and  2008  are  explicitly  modeled  As  such.  Value  Added  Coefficients  are 
computed  for  those  years  only.  The  VALOR  Model  requires  objective  coefficients  for 
each  system  for  each  year  the  system  can  be  procured.  Ideally  each  year  in  the  period 
under  consideration  would  be  modeled  explicitly.  Lacking  that,  the  assumption  was 
made  that  the  effectiveness  would  vary  linearly  between  the  years  that  explicit 
modeling  is  performed.  Thus,  linear  interpolation  is  used  to  calculate  the  objective 
coefficients  for  the  years  that  are  not  explicitly  modeled. 

(3)  Form  of  the  Objective  Function.  The  objective  function  of  the  VALOR 
Model  can  be  described  as  follows.  Let  Vy  be  the  per-item  contribution  of  the  system 
to  force  effectiveness,  and  let  x^;  be  defined  as  the  quantity,  of  system  i  procured  in 
year  j,  where  j  =  1,...,  n,  with  n  being  the  number  of  years  in  the  planning  horizon. 
For  the  VAA  Phase  II  Study,  n  =  15.  The  objective  function  can  then  be  written  as 


n  m 

\laximizp  ^  y  x 


In  the  analysis,  m  systems  will  be  considered.  For  the  VAA  Phase  II  Study,  m  =  41. 
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c.  Budg  etary  Constraints.  Note  at  the  outset  that  there  is  a  separate  budgetary 
constraint  for  each  year  in  the  time  period  of  interest  for  the  study.  Funds  designated 
for  use  in  a  particular  year  cannot  be  carried  over  into  following  years.  The  model 
will  maximize  effectiveness  which  is  accumulated  in  the  objective  by  the 
procurement  of  equipment  through  the  expenditure  of  funds.  The  funds  available  in 
each  year  will  be  specified  in  these  budgetary  constraints,  so  their  accuracy  is 
extremely  important  to  obtaining  a  valid  solution. 

(1)  Total  Obligational  Authority  (TOA).  As  discussed  in  Chapter  7,  the  TOA 
that  can  be  spent  on  procuring  the  systems  considered  in  the  VAA  Phase  11  Study  is 
only  a  percentage  of  the  entire  RDA  budget.  This  amount,  Bj,  gives  the  constraint  on 
how  much  can  be  spent  on  the  candidate  systems  in  each  year,  and  it  will  be  the  right- 
hand  side  of  the  budgetary  constraint. 

(2)  Learning  Curve  Costs.  The  VALOR  Model  handles  fixed  costs,  both 
RDTE  and  fixed  manufacturing  costs,  linear  variable  costs  or  average  unit  costs,  and 
nonlinear  learning  curve  costs.  The  method  for  dealing  with  learning  curve  costs  is 
discussed  in  this  subparagraph.  Using  the  power  form  of  the  learning  curve,  the 
following  is  an  expression  for  the  average  unit  cost  for  system  i  in  year],  when  the 
quantity  manufactured  in  year  j  is  considered  as  the  "lot"  for  that  year. 


where: 


F  ■^-L  +2(F  L 

Q  =  lot  midpoint  =  — - - - - — ~ — 

with  Fy  =  accumulated  number  of  the  first  item  of  type  i  produced  in  year  j,  Ly  = 
accumulated  number  of  the  last  item  of  type  i  produced  in  year  j, 

Ai  =  the  first  unit  cost  of  system  i,  and  bi  =  the  learning  curve  slope  parameter  for 
system  i.  Notice  that  Cy  is  a  function  of  the  quantity  of  system  i  produced  in  year  j, 
xy,  as  well  as  the  quantities  produced  in  every  year  through  year  j  -  1.  Thus,  the 
learning  curve  cost  term  is  both  nonlinear  and  not  separable.  For  this  reason,  an 
approximation  was  devised  to  represent  these  learning  curve  cost  terms  as  a 
piecewise  linear  approximation  to  a  separable  cost  function.  Thus,  each  of  these 
learning  curve  cost  terms,  cyxy,  gives  rise  to  several  new  terms  in  each  budgetary 
constraint,  as  well  as  several  new  constraints  involving  binary  variables.  The 
inclusion  of  this  feature  in  the  VALOR  Model  greatly  increases  the  complexity  of  the 
mathematical  programing  problem  (see  CAA-TP-92-1  for  details). 

(3)  Nonlearning  Curve  Variable  Costs.  Not  all  of  the  systems  considered  in 
this  study  exhibit  learning  behavior.  For  these  systems,  an  average  unit  cost,  cy,  is 
specified  for  each  year  of  production.  The  cost  term  associated  with  these  systems 
have  the  form  CyXy,  for  each  yea-,  j,  that  system  i  is  produced.  These  terms  are  then 
included  in  the  budgetary  constraints  for  appropriate  years.  Some  systems  have  a 
component  of  cost  that  is  more  appropriately  described  as  a  "nonlearning"  variable 
cost.  For  these  instances,  the  cost  of  the  system  can  be  described  as  having  both  a 
learning  and  nonlearning  component,  and  the  term  cyxy  can  be  introduced  together 
with  the  learning  cost  in  the  budgetary  constraint. 
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(4)  Considera  tion  of  F’ixed  Costs.  Some  costs,  such  as  RDTE,  Rij,  expendi¬ 
tures,  or  nonrecurring  fixed  manufacturing  costs,  Vij,  are  not  incurred  on  a  per-unit 
basis.  These  costs  simply  reduce  the  funds  available  for  procurement  (lower  the 
values  of  the  Bj)  over  the  years  they  are  expended.  However,  when  the  model  is  used 
to  evaluate  cuts  or  cancellation  of  programs,  significant  savings  can  be  accrued  by 
recouping  the  RDTE  or  other  fixed  cost  funds  that  have  not  yet  been  spent.  Thus, 
these  costs  must  be  tied  to  the  programs  being  evaluated  in  a  meaningful  way.  The 
discussion  of  how  to  handle  the  evaluation  of  potential  program  cancellations  will 
appear  later  in  this  chapter. 

d.  Force  Structure  Constraints.  Force  structure  requirements  drive  the 
decisions  of  how  many  of  a  particular  item  of  equipment  should  be  procured.  The 
study  sponsor  must  specify  the  level  of  force  structure  to  be  considered.  For  example, 
the  sponsor  might  know  that  insufficient  funds  are  available  to  buy  enough  for  all  the 
force  packages,  so  he  may  designate  that  Force  Package  I,  the  highest  priority  group 
of  units  in  the  force,  must  be  filled  with  equipment  for  all  systems,  and  that  nothing 
would  be  procured  beyond  those  needed  to  fill  Force  Package  11,  the  next  highest 
priority  group  of  units.  Thus,  the  force  structure  bounds  for  each  system  would  be 
established.  It  would  then  be  known  what  the  minimum  and  maximum  allowable 
procurements  are  for  each  system  by  the  end  of  each  system's  production  campaign. 
For  most  of  VAA  Phase  II,  the  sponsors  specified  that  the  Force  Package  I  quantities 
would  be  procured  for  any  system  that  was  recommended  to  be  procured  at  all.  Later 
in  the  study,  a  range  was  specified.  Also,  there  are  two  schemes  for  specifying 
constraints  on  yearly  procurement  quantities.  These  schemes  are  discussed  below. 

(1)  Equal  Quantity  Representation.  Of  the  two  ways  to  represent  the  force 
structure  constraints  in  the  optimization  model,  the  first  involves  constraining  each 
year's  procurement  quantity  of  each  system  to  be  the  following: 


-  <  X  <  - ,  V  systems  i produced  in  vea  r j,  V 

n  ‘J  n  ~  J 

I  1 


where,  for  system  i, 

Fimin  =  minimum  force  Structure  requirement, 

Fimax  =  maximum  force  structure  requirement, 
and  ni  =  number  of  years  of  production. 

The  advantage  of  thi  lethod  is  that  the  series  of  production  quantities  over  the 
production  campaign  tends  to  be  more  stable.  From  a  practical  standpoint,  avoiding 
wide  swings  in  annual  production  quantities  is  desirable.  The  disad\  antage  is  that 
the  constraint  is  tighter,  limiting  the  flexibility  of  the  model  to  find  a  better  solution. 
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(2)  Total  Quantity  Representation.  The  second  option  involves  constraining 
the  sum  of  all  items  produced  over  the  entire  production  campaign  to  be  between  the 
force  structure  minimum  and  maximum.  That  is, 


r  +  n.  —  I 

t  i 

F  ^  \  S  F  V 

I  l,J. 

min  /  =  i  max, 

where  ti  =  the  first  year  of  production  for  system  i.  Constraints  on  the  individual  xij’s 
would  be  based  on  production  capacities.  These  constraints  will  be  discussed  later. 
This  option  allows  for  better  solutions  with  respect  to  the  effectiveness  objective,  but 
tends  to  give  wide  swings  in  the  annual  production  quantities.  The  decision 
regarding  which  of  these  schemes  to  use  rests  with  the  study  sponsor.  Note  that  the 
choice  of  schemes  can  be  made  on  a  system-by-system  basis,  thus  maximizing  the 
flexibility  of  the  model.  A  modification  of  this  representation  is  the  introduction  of 
additional  constraints  on  the  Xi/s  that  would  be  used  to  force  a  more  stable  stream  of 
production  quantities.  Letting  Pj  be  the  allowable  variation  in  yearly  production 
quantities,  and  noting  that  these  constraints  would  not  be  applied  during  ramp-up 
years,  these  constraints  would  take  the  form 


(1-p, 


iX,  < 

llj  -  1)  IJ 


■^1  l(J  -  1)  I 


V  +  1 


I  I 


-  1  1 


In  this  case,  the  production  quantity  would  be  constrained  to  be  within  Pj  percent  of 
the  previous  year's  quantity,  avoiding  undesirable  swings  in  production.  Of  course, 
the  selection  of  the  percentage  is  arbitrary  and  can  be  adjusted  appropriately  on  a 
system  by  system  basis. 

e.  Production  Constraints.  Thecapacitiesofthe  various  production  facilities 
must  be  considered  in  constraining  the  production  quantities.  Typically,  the  model  is 
constrained  to  produce  at  least  the  MSR  of  production  and  no  more  than  the  MPR. 

The  MSR  is  defined  as  the  production  rate  needed  to  keep  the  production  line  open 
while  maintaining  a  responsive  vendor  and  supplier  base.  The  MSR  is  frequently 
equated  to  maintaining  a  warm  production  base.  The  MPR  is  the  production  rate 
which  maximizes  the  production  capacity  of  existing  tooling  or  facilities  without 
requiring  additional  investment  to  increase  the  capacity.  Most  systems  that  are  not 
yet  in  production  at  the  beginning  of  the  planning  horizon  will  have  a  ramp-up  in 
production  over  2  or  3  years  where  the  production  capacity  is  lower  than  the  MSR. 
The  production  constraints  must  be  used  in  conjunction  with  the  force  structure 
constraints  to  develop  the  upper  and  lower  bounds  of  the  production  quantities,  x^j. 
When  using  the  equal  quantity  representation  of  the  force  structure  bounds,  some 
preprocessing  is  required  to  ensure  that  these  bounds  are  applied  appropriately.  Let 

Pminij  =  lower  production  bound  (MSR)  for  system  i  in  year  j, 

Pmaxij  =  upper  production  bound  (MPR)  for  system  i  in  year  j, 

Fminij  =  lower  force  structure  bound  for  system  i  in  year  j, 

Fmaxy  =  upper  force  structure  bound  for  system  i  in  year  j, 

Uij  =  upper  hound  on  Xy  for  system  i  in  year  j,  and 
Lij  =  lower  bound  on  xij  for  system  i  in  year  j. 
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The  upper  and  lower  bounds  on  Xy  are  determined  using  the  following  rules. 

(1)  IfPminij  ^  Fminij,  then  Lij  =  Pminy. 

(2)  If  Fminij  =  Pminij,  then  Lij  =  Fminij. 

(3)  IfPmaxij  ^  Fmaxij,  then  Uij  =  Fmaxij. 

(4)  IfFmaxij  ^  Pmaxy,  then  Uij  =  Pmaxij. 

(5)  IfFmaxij  ^  Pmintj,  then  Ly  -  Uij  =  Pminij. 

(6)  If  Fminij  ^  Pmaxy,  then  Ly  =  Uij  =  Pmaxy. 

Note  that  rules  (5)  and  (6)  address  the  problem  of  mismatch  between  the  production 
limitations  and  the  force  structure  requirements.  When  these  conditions  arise,  it  is 
prudent  to  reconsider  the  length  of  the  production  campaign.  Note  also  that  the 
above  rules  only  apply  to  the  first  alternative  for  determining  force  structure 
constraints.  If  the  second  alternative  is  employed,  namely  the  total  quantity 
representation,  the  production  bounds  are  used  to  bound  xy's.  Care  must  be  taken, 
however,  to  ensure  that  mismatches  in  the  production  limitations  and  force  structure 
requirements  are  resolved.  Otherwise,  an  infeasible  program  will  result 

f.  Evaluation  of  Program  Cuts.  As  mentioned  previously,  the  model  needs  to 
be  useful  in  analyzing  program  cut  alternatives  which  arise  during  the  building  of 
the  Army  program  and  budget.  In  order  to  do  so,  two  factors  must  be  considered. 
First,  the  procurement  of  either  the  quantity  of  a  certain  system  that  conforms  to  the 
above  constraints  or  a  zero  quantity  must  be  feasible.  Next,  if  some  fixed  costs,  such 
as  RDTE  funds,  are  associated  with  the  program,  independent  of  the  quantity 
purchased,  then  they  would  also  be  saved  if  the  program  is  canceled. 

(1)  Implementation.  In  order  to  implement  this  enhancement  to  the  model, 
binary  variables  were  introduced  and  defined  as  follows: 


U.  = 


1 ,  if  system  i  is  procured, 
0,  otherwise 


Then  modify  the  constraints  on  the  Xy’s  as  follows. 

EyUi  ^  Xy  =  UijUi. 

Note  that  when  system  i  is  procured,  the  Uj  =  1,  and  these  constraints  become 
equivalent  to  those  previously  discussed.  When  ui  =  0,  then  the  value  of  Xy  is 
constrained  to  be  zero.  These  binary  variables  serve  two  additional  purposes.  The 
first  is  as  a  switch  which  turns  the  fixed  costs  on  and  off  in  the  budget  constraints, 
and  the  second  is  as  a  mechanism  to  relate  system  procurements. 
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(2)  Fixed  Costs.  If  a  system  is  excluded  from  the  list  of  systems  recommended 
for  procurement,  its  ui  variable  will  be  set  at  0.  Thus,  by  including  a  term  in  the 
budget  constraint  for  year  j  that  represents  the  expenditure  of  some  fixed  costs  for 
system  i,  e.g.,  RijUj,  the  fixed  cost  will  only  be  assessed  against  the  budget  constraint 
if  Ui  =  1.  So  these  fixed  costs  do  not  come  into  play  unless  the  system  is  on  the 
recommended  list. 

(3)  Related  System  Procurements.  The  other  important  use  of  these  binary 
Ui  variables  is  to  constrain  the  model  to  procure  systems  in  combinations.  For 
example,  suppose  the  procurement  of  a  new  artillery  system  and  a  resupply  vehicle 
that  will  carry  its  ammunition  is  under  consideration.  It  may  be  necessary  to 
constrain  the  model  to  refrain  from  procuring  the  resupply  vehicle  unless  it  procures 
the  artillery  system  as  well.  By  introducing  the  following  constraint,  the  model  can 
be  forced  to  relate  these  systems  as  described  above. 

^resupply  '  ^artillery  —  0. 

This  constraint  ensures  that  the  binary  variable  associated  with  the  artillery  system 
is  greater  than  or  equal  to  that  of  the  resupply  vehicle.  Thus,  the  unreasonable  result 
of  procuring  the  resupply  vehicle  without  the  artillery  system  is  avoided.  The 
introduction  of  this  type  of  constraint  in  various  ways  allows  the  analyst  to  evaluate 
different  combinations  of  systems.  Also,  by  constraining  one  of  these  variables  to  be 
either  1  or  0,  the  analyst  can  either  force  a  system  into  or  out  of  the  solution, 
respectively. 

8-3.  IMPLEMENTATION  AND  PERFORMANCE 

a.  Hardware  and  Software.  The  VALOR  Model  was  implemented  using  the 
IBM  Optimization  Software  Library  (OSL),  on  an  IBM  RISC  6000  work  station.  OSL, 
as  the  name  suggests,  is  a  set  of  subroutines  that  can  be  called  to  manipulate  and 
solve  a  variety  of  optimization  problems.  It  can  be  accessed  through  FORTRAN  or  C 
programing  languages.  The  software  is  flexible  with  respect  to  input  and  accepts 
either  standard  MPS  input  format  or  allows  direct  access  to  the  data  structure  to  set 
up  the  problem. 

b.  Programming.  A  front-end  application  program  in  FORTRAN  that  reads  the 
data,  processes  the  data  into  the  appropriate  data  structures  for  the  optimizer,  calls 
the  optimization  subroutines,  and  then  prints  the  results  was  written  to  implement 
the  model. 

c.  Performance 

(1)  Early  Tests.  The  first  test  problems  with  realistic  data  were  of  the  order  of 
700  rows  with  800  variables,  of  which  350  were  binary.  This  program  evaluated 
approximately  20  systems  and  took  almost  20  minutes  of  CPU  time  to  run.  The 
model  was  also  run  using  the  system  cutting  feature,  the  running  time  increased  25  - 
50  percent.  In  these  early  tests,  only  a  small  subset  of  the  systems  was  evaluated  as 
candidates  for  elimination  from  the  Army  budget,  usually  three  or  four  systems.  Also 
of  note  was  that  attempting  to  evaluate  too  many  such  systems  greatly  increased  the 
run  time.  Although  the  run  time  was  increased  for  most  of  the  runs  in  which  this 
feature  was  employed,  there  were  instances  where  the  increase  in  run  time  was 
minimal. 
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(2)  Performance  Tuning.  Several  efforts  were  undertaken  to  improve  the 
performance  of  the  VALOR  Model.  The  first  entailed  preprocessing  the  program 
using  the  OSL  preprocessing  subroutines.  This  effort  had  negative  results,  since  the 
preprocessor  could  not  improve  the  performance  of  the  model,  while  additional  time 
was  needed  to  perform  the  preprocessing  step.  Next,  efforts  were  undertaken  to 
identify  special  structure  in  the  formulation  that  could  be  exploited.  Two  classes  of 
strong  inequalities  were  found,  which  were  already  implicit  in  the  formulation,  but 
whose  explicit  inclusion  vastly  improved  the  speed  of  solution.  Final  test  runs,  as 
well  as  the  production  runs,  were  significantly  larger  than  the  initials  tests  that  were 
performed.  Ultimately,  41  systems  were  analyzed,  of  which  22  had  learning  curve 
costs.  The  mixed  integer  program  had  about  4,000  rows  with  3,000  variables,  of 
which  about  500  were  binary  integers  and  5,500  nonzero  elements.  The  run  time  for 
this  improved  formulation  was  reduced  to  between  2  and  13  minutes  of  CPU  time  on 
the  IBM  RISC  6000  320H.  Another  important  aspect  of  this  improved  formulation 
was  that  limits  on  evaluating  systems  as  candidates  for  elimination  were  removed. 
This  capability  was  established  to  make  the  procurement  of  all  the  systems  optional. 
This  capability  became  very  important  in  the  conduct  of  the  Value  Added  Analysis, 
since  the  main  emphasis  was  on  identifying  funding  tradeoffs  among  the  candidate 
systems.  Without  the  performance  enhancing  modifications,  the  model  would  not 
have  been  as  responsive  as  was  necessary  to  provide  the  required  analytical  support. 

8-4.  RESULTS  AND  CONCI.LSIONS 

a.  Methodology  and  Computer  Resources.  The  methodology  introduced  in 
this  chapter  seems  to  do  a  good  job  of  incorporating  the  learning  curve  effects  on 
costing  into  the  budget  constraints  of  the  Value  Added  Analysis  acquisition  strategy 
optimization.  The  introduction  of  this  feature  greatly  increases  the  computational 
overhead  associated  with  solving  problems  of  this  nature.  As  a  result, 
implementation  of  this  enhancement  to  acquisition  strategy  models  requires 
significantly  increased  computing  resources  to  obtain  a  solution. 

b.  App  roximation.  This  methodology  is  an  approximation,  and  checks  are 
necessary  to  ensure  the  approximation  is  accurate  enough.  Experience  has  shown 
that  the  approximation  has  yielded  results  in  which  the  expended  program  dollars, 
calculated  using  the  nonlinear  cost  function  and  the  optimized  quantities,  are  within 
2  percent  of  the  nominal  value.  Considering  the  approximate  nature  of  costing 
systems  that  will  only  be  procured  in  the  far  distant  future,  2  percent  is  adequate.  In 
applications  that  require  more  accuracy,  the  approximation  can  be  made  more  exact 
by  increasing  the  number  of  pieces  in  the  piecewise  approximation. 

c.  Application.  The  use  of  this  methodology  has  been  shown  to  improve  the 
quality  of  the  optimization  for  the  purpose  of  acquisition  strategy.  In  this  era  of 
tightly  constrained  budgets  for  procurement,  accurate  costing  is  essential  to  get  the 
most  from  limited  funds.  This  methodology  has  enhanced  analytical  efforts  that  help 
accomplish  this  task.  This  optimization  model  was  successfully  used  to  assist  the 
Army  Staff  in  evaluating  the  various  alternative  weapon  systems  considered  for 
procurement.  The  model  was  particularly  useful  in  identifying  the  years  in  which 
budget  constraints  were  extremely  tight  with  respect  to  planned  production 
campaigns,  suggesting  modifications  that  could  be  made  to  proposed  programs.  The 
model  was  also  extremely  useful  in  identifying  systems  that  were  excluded  from  the 
solution  when  other  systems,  or  combinations  of  systems,  were  forced  to  be  procured. 
This  capability  gave  the  leadership  a  window  into  the  cost  of  their  decisions  as  they 
related  to  system  tradeoffs. 
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d.  Conclusion.  VALOR  has  provided  a  new  dimension  to  the  PPBES  process  for 
the  Department  of  the  Army  Stan.  The  Staff  now  has  available  in  a  single  model  the 
capability  to  pull  together  data,  policy,  and  guidance  quickly  and  accurately  in  order 
to  develop  a  balanced  Army  program. 
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CHAPTERS 

SUMMARY  AND  FINDINGS 


9-1.  INTRODUCTION.  The  purpose  of  this  chapter  is  to  provide  the  summary  and 
findings  for  the  Value  Added  Analysis  (VAA)  Phase  IT  Study.  Chapter  9  is  structured 
as  follows: 

•  Introduction 

•  Summary 

•  Observations  and  Findings 

•  Quick  Reaction  Analyses  Completed 

•  Future  Development 

9-2.  SUMMARY.  The  purpose  of  the  VAA  Phase  n  Study  was  to  implement  the 
VAA  methodology  as  created  and  defined  in  the  Phase  I  work.  There  has  been  partial 
success  in  implementing  the  Value  Added  methodology  as  originally  envisioned.  A 
primary  objective  of  the  VAA  Phase  II  Study  was  to  automate  a  large  portion  of  the 
methodology  for  use  by  action  officers  in  the  Pentagon.  The  concept  of  action  officers 
conducting  a  full-scale  value  added  type  analysis  proved  to  be  unrealistic  as  a  result 
of  the  work  completed  in  this  study.  The  combat  modeling,  the  structuring  of  costs, 
the  formulating  of  the  optimization,  and  the  defining  of  issues  requires  a  level  of 
commitment  that  the  action  officers  in  the  Pentagon  cannot  provide.  The  daily 
requirements  of  the  action  officer  do  not  allow  the  time  to  conduct  these  types  of 
analyses  "in  house."  A  major  finding  as  a  result  of  this  issue  is  the  need  to  have  an 
analytical  organization,  such  as  the  US  Army  Concepts  Analysis  Agency,  involved  in 
the  process  throughout  the  analysis  period.  Furthermore,  this  issue  of  time  and 
resource  availability  points  to  a  need  to  modify  the  architecture  and  data  structure  of 
the  VAAC.  The  following  paragraph  provides  specific  observations  and  findings. 

9-3.  OBSERVATIONS  AND  FINDINGS 

a.  Essential  Fllements  of  Analysis  Observations  and  Findings.  The 
observations  and  findings  contained  in  this  paragraph  are  keyed  to  the  EEA  as  listed 
in  the  study  directive. 

(1)  EEA  1,  What  is  the  "value  added"  of  a  select  number  of  major  item 
systems  to  be  considered  in  the  1994-1999  Program  Objective  Memorandum 
(POM)? 
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(a)  The  team  initially  started  with  a  list  of  1 19  systems  for  review  as  part  of 
the  VAA  Phase  11  effort.  This  list  was  revised  using  a  set  of  criteria  which  included 
the  team’s  ability  to  model  the  system,  the  existence  of  a  system  requirements 
document,  and  the  determination  that  the  system  represented  a  major  acquisition  in 
terms  of  the  total  dollars  associated  with  the  system's  program.  The  first  cut  of  the 
original  1 19,  based  on  the  previously  mentioned  criteria,  resulted  in  a  list  of  50  major 
item  systems.  Each  of  the  systems  on  this  "first  cut"  list  were  modeled  in  CORE  AN. 
As  the  team  gained  experience  with  particular  systems  and  as  decisions  were  made 
by  the  Army  leadership,  several  systems  were  dropped  from  consideration.  The  final 
list  consisted  of  41  systems  as  shown  in  Figure  9-1  under  the  2008  column. 

(b)  Figure  9-1  provides  the  "value  added"  of  each  of  the  systems  by  year  (FY 
96,  FY  01,  and  FY  08)  as  a  1  to  N  list  developed  from  the  Effectiveness  Integration 
Module.  A  word  of  caution  is  provided  for  the  reader  at  this  point.  These  numbers 
represent  the  combined  effectiveness  for  both  the  Stressful  and  Contingency 
Scenarios  and  prior  to  their  modification  as  per-system  values.  The  values  actually 
represent  the  effectiveness  of  the  force  as  modeled  in  CORBAN.  The  number  is  a 
total  for  actual  numbers  played  and  not  necessarily  for  the  final  optimized  number  of 
systems. 

(2)  EEA2.  What  kind  of  characteristics  should  the  VAAC  have  in  order 
to  implement  the  VAA  methodology? 

(a)  As  defined  in  the  VAA  Phase  I  Study,  the  VAAC  was  to  be  a  set  of  models 
and  tools  residing  on  the  METAPHOR  computer  architecture.  The  advantage  of  this 
arrangement  as  originally  developed  was  that  it  would  allow  the  action  officer  in  the 
Pentagon  to  use  the  data  base  engine  of  METAPHOR  to  define  and  shape  the  issues 
to  be  studied.  The  action  officer  would  then  use  the  ability  of  METAPHOR  to  move  in 
and  out  of  the  PC  environment  in  order  to  run  the  VAA  models  and  tools  for 
conducting  a  value  added  analysis.  CAA,  PAED,  and  ODCSOPS  Information 
Management  Office  (IMO)  were  able  to  obtain  connectivity,  and  many  of  the  tools  and 
unique  VAA  data  bases  were  placed  in  the  METAPHOR  environment.  However,  the 
ability  to  move  in  and  out  of  the  METAPHOR  environment  into  the  PC  environment 
was  found  to  be  somewhat  limited.  This  limitation,  coupled  with  time  and  resource 
constraints  associated  with  having  action  officers  attempt  to  do  these  types  of 
analyses,  prohibited  the  implementation  of  the  proposed  VAAC  design  during  the 
study  period.  As  such,  the  VAA  methodology  as  described  provides  the  appropriate 
VAAC,  but  the  actual  work  should  be  accomplished  by  an  analytical  organization 
such  as  CAA. 
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(b)  The  team  found  that  the  action  officers  in  the  Pentagon  were  unable  to 
find  the  time  to  learn  either  the  METAPHOR  computer  software  or  the  VAA  tools. 
This  constraint  indicated  to  the  VAA  study  team  that  the  VAAC  would  have  to  be 
designed  to  facilitate  the  action  officers  having  access  to  the  input  data,  access  to  the 
issue  definition  narrative  data,  and  then  to  the  results  of  the  Value  Added  Analysis. 
However,  the  Value  Added  Analysis  would  need  to  be  conducted  by  an  analytical 
organization  outside  of  the  HQD  A  ARSTAF  or  by  a  special  study  group.  The  need  to 
have  a  separate  study  group  and  visibility/access  for  the  action  officers  creates  the 
requirement  for  the  VAAC  to  be  able  to  move  easily  from  a  data  base  environment 
into  a  decentralized  computational  environment  and  back  again.  The  ability  to  pull 
data  from  the  Army  corporate  data  bases,  format  the  data,  and  then  move  that  data 
into  the  VAA  tools  and  models  must  be  well  developed.  The  connectivity  of  data  and 
tools  must  be  enhanced  over  the  manner  in  which  this  type  of  function  was  handled  in 
VAA  Phase  El,  which  was  the  hand-carrying  of  data  on  floppies  from  one  machine  to 
another.  The  study  group  conducting  the  analysis  should  be  able  to  move  from  the 
IBM  RISC  6000  computer  to  the  Macintosh  computer  and  back  to  the  data  bases  in  a 
manner  which  is  transparent  to  the  analyst.  Likewise,  the  action  officer  must  be  able 
to  query  the  results  of  the  study  group  work,  the  assumptions  for  the  work,  and  the 
input  data  for  the  particular  work. 

(3)  EEA3.  What  kinds  of  tools,  techniques,  and  models  are  needed  to 
support  the  VAA  methodolog3'? 

(a)  The  VAA  Phase  I  effort  investigated  the  idea  of  using  spreadsheet  costing 
tools,  a  simplified  optimization,  and  the  use  of  CORE  AN  for  the  combat  modeling. 
Much  of  the  work  that  was  completed  during  Phase  I  was  hand-generated,  with  little 
experience  as  to  how  data  or  processes  might  be  connected.  The  major  thrust  of  the 
VAA  Phase  n  effort  was  to  develop  a  more  complete  combat  modeling  capability, 
expand  the  optimization  tool,  implement  the  changes  to  TOPSIS  recommended  by  the 
ORSA  Center  at  the  USMA,  and  take  the  analysis  concepts  and  develop  them  into  a 
full  set  of  tools.  The  survey  technique  developed  during  VAA  Phase  I  and  the  use  of 
the  analytical  hierarchy  process  remained  relatively  unchanged.  The  only  change 
associated  with  these  last  two  techniques  was  the  development  of  a  different 
hierarchy.  This  was  not  a  matter  of  change  in  technique  as  much  as  it  was  defining 
the  context  for  the  VAA  Phase  El  effort.  Essentially,  the  survey  approach  was  the 
same  in  both  phases. 

(b)  The  first  model  improvement  needed  to  support  the  VAA  methodology 
included  a  combat  model  which  was  both  easier  to  set  up  and  faster  in  its  run  time 
than  the  stochastic  CORBAN  used  in  the  Phase  I  study.  The  team  looked  at  other 
models  such  as  AFP,  Eagle,  and  VIC  as  possible  solutions.  The  team  decided  that 
AFP  was  at  too  aggregated  a  level,  that  Eagle  was  not  yet  developed  enough  to  use, 
and  that  VIC  was  at  too  high  a  level  of  resolution.  All  of  these  models  also  had  the 
same  limitations  as  CORBAN  in  that  they  all  have  extensive  setup  requirements. 
This  review  of  the  combat  models  had  the  net  effect  of  validating  the  need  for 
CORBAN.  However,  the  team  determined  that  a  full  experimental  design  using 
stochastic  CORBAN  was  not  possible.  Therefore,  the  team  worked  with  TRADOC  to 
obtain  a  deterministic  version  of  CORBAN  in  an  effort  to  speed  up  the  run  time.  This 
change  proved  to  be  extremely  important,  because  it  allowed  the  team  to  perform 
excursions  relatively  rapidly.  However,  in  order  to  develop  values  for  all  41  systems, 
a  very  large  experimental  design  was  required.  Even  a  deterministic  CORBAN  was 
not  fast  enough  to  accomplish  all  the  runs  required.  The  requirement  to  turn  over 
many  excursions  for  a  wide  range  of  weapon  system  mixes  led  to  the  use  of  an  RSM. 
The  VAA  Phase  El  experience  in  using  RSM  was  very  favorable.  The  RSM  technique 
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allowed  the  team  to  accomplish  the  development  of  values  for  all  of  the  systems.  RSM 
also  had  a  side  benefit  of  allowing  the  team  to  predict  the  explicit  MOE  values  one 
would  obtain  by  playing  the  weapon  mix  coming  out  of  the  optimization. 

(c)  A  more  fully  developed  optimization  formulation  was  another 
technique/model  needed  to  fully  implement  the  VAA  methodology.  In  VAA  Phase  I, 
a  relatively  simple  formulation  was  used  for  the  linear  program  in  order  to  demon¬ 
strate  the  feasibility  of  the  methodology.  It  was  evident  that  any  follow-on  effort  had 
a  need  for  dynamic  costing  to  accommodate  the  concept  of  system  learning  curves  in 
order  to  provide  a  more  accurate  cost  within  the  optimization.  Phase  11  also  identified 
the  requirement  to  link  certain  systems  together  if  they  are  dependent  upon  one 
another.  For  example,  in  the  AFAS  and  FARV-A  case,  FARV-A  is  not  procured 
unless  AFAS  is  being  procured.  Finally,  the  capacity  to  continue  to  fully  incorporate 
both  force  structure  and  production  data  was  maintained,  along  with  institution  of 
better  quality  control  on  the  data. 

(d)  VAA  Phase  I  validated  the  concept  of  using  a  linear  combination 
technique  for  integrating  the  effectiveness  values.  The  TOPSIS  technique  used  in 
Phase  I  was  reviewed  by  the  ORSA  Center  at  USMA.  Their  suggestions  for 
improvement  were  implemented  in  the  Phase  El  Study. 

(e)  The  cost  analysis  tools  developed  as  part  of  this  VAA  Phase  11  effort  have 
proven  to  be  very  useful  to  the  sponsors.  LCCM  has  been  expanded  and  refined  to 
include  the  ability  to  conduct  "what  ifs"  on  quantities  and  average  unit  costs.  The 
tool  has  been  transferred  to  HQDA  PAED  and  is  currently  in  use  by  PAED  analysts. 
Further  refinement,  to  include  the  development  of  generic  LCCMs  by  mission  area, 
would  be  useful  in  conducting  analysis  on  developmental  systems  which  currently  do 
not  have  a  BCE. 

(4)  EE  A  4.  What  explicit  effectiveness  models  are  most  appropriate  for  a 
Value  Added  Analysis  Study  (Value  Added  Analysis  1994-1999)? 

(a)  In  VAA  Phase  I,  the  issue  of  using  a  highly  aggregated  combat  model  was 
addressed.  The  conclusion  was  that  an  aggregated  model  could  provide  useful 
insights  for  answering  macro-level  program  tradeoff  questions.  However,  it  was  also 
noted  that  decisionmakers  require  sufficient  detail  to  understand  the  reasons  for 
combat  outcomes  to  assist  them  in  specific  program  decisions.  This  point  was 
reinforced  in  work  completed  as  part  of  VAA  Phase  H.  Almost  universally, 
decisionmakers  wanted  to  know  the  reasons  behind  the  combat  results.  The  use  of 
RSM  compounded  the  problem  by  not  considering  the  effects  of  interaction  and  only 
looking  at  the  first  order  effects.  This  tradeoff  between  detail  and  the  ability  to  look 
at  a  large  number  of  systems  will  continue  to  be  an  issue. 

(b)  It  appears  to  the  study  team  that  for  weapons  systems'  tradeoffs,  the 
force-level  model  continues  to  be  appropriate.  However,  the  issue  of  interaction  and 
need  to  explain  results  reinforces  an  idea  from  the  Phase  I  Study  regarding  a 
hierarchy  of  models  and  MOE.  The  team  was  unable  to  explore  this  concept  in  any 
detail  at  this  time.  It  is  recommended  that  the  idea  of  model/MOE  hierarchy  and  how 
one  would  incorporate  the  RSM  design  into  this  hierarchy  be  research  as  part  of  a 
follow-on  study. 
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(c)  This  particular  EE  A  was  aimed  at  a  larger  issue  than  just  RD  A 
appropriation  modeling  and  the  combat  modeling  associated  with  study  weapon 
system  tradeoffs.  At  the  outset,  VAA  Phase  11  was  starting  to  look  at  other 
appropriations  such  as  OMA,  MCA,  and  perhaps  MPA.  Although  these 
appropriations  were  considered,  especially  in  the  LCCM  costing,  true  tradeofYs  were 
not  conducted.  The  team  did  explore  some  preliminary  tradeoffs  in  the  area  of  OMA 
to  RDA  and  MPA  to  RDA.  This  preliminary  work  suggests  that  additional  explicit 
effectiveness  models  are  required,  such  as  Operational  Tempo  (OPTEMPO)  models, 
training  models,  and  the  inclusion  of  campaign  models.  This,  too,  is  an  area  which 
requires  additional  research  as  part  of  a  follow-on  study. 

(5)  EEA  5.  What  refinements  are  needed  to  improve  the  VAA 
methodology?  Answers  to  the  previous  EEA  suggest  some  areas  for  refinements. 
The  work  completed  in  the  VAA  Phase  TI  effort  has  show  n  that  the  methodology 
works  and  can  provide  useful  and  timely  support  to  decisionmakers.  Additional  work 
is  needed  to  expand  the  use  of  the  methodology  into  a  broader  range  of  issues.  These 
include  a  variety  of  combat  systems  (e.g.,  logistics  and  command  and  control  (C-'j)  and 
other  appropriations  (e.g.,  OMA  and  MCA). 

b.  Other  Observations  and  Findings.  The  observations  and  findings  contained 
in  this  paragraph  are  not  necessarily  keyed  to  the  original  questions  proposed  in  the 
study  directive,  but  are  insights  obtained  as  a  result  of  the  work  completed  in  this 
study. 

(1)  Input  data  intensity,  accuracy,  and  timeliness  continue  to  be  of  crucial 
importance  in  making  the  value  added  process  operational. 

(2)  The  need  to  have  the  survey  process  standardized  was  confirmed  in  this 
study. 

(3)  Issue  definition  is  clearly  an  iterative  process  and  should  not  be  thought  of 
as  having  a  beginning  or  ending,  per  se. 

(4)  Future  experimental  designs  should  try  to  incorporate  the  need  to 
investigate  second  order  effects  (i.e.,  synergism  and  weapon  system  interactions). 

(5)  Production  data  as  depicted  in  the  procurement  (P-21)  forms  is  not  sufficient 
or  timely  enough  to  conduct  Value  Added  Analysis  on  an  institutionalized  basis.  The 
production  data  required  for  Value  Added  Analysis  (as  well  as  for  other  acquisition 
analyses  conducted  for  HQDA)  should  be  made  readily  available  and  current  in 
support  of  budgetary  decisionmaking. 

(6)  Standardized  cost  codes  and  views  as  implemented  in  the  VAA  Phase  II 
Value  Added  Analysis  Study  should  be  used  throughout  the  Army.  The  Life  (^ycle 
Cost  Model  should  be  made  available  to  program  e.xecutive  offices  (PEOs)^PMs  and 
help  form  the  basis  for  standardization. 

(7)  Major  item  system  data  as  defined  in  the  Army  Resource  Integration  and 
Management  (ARIM)  Study  should  be  institutionalized  and  used  throughout  to  assist 
in  standardization  of  costs  and  system  definition. 
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9-4.  QUICK  REACTION  ANALYSES  (QRAs)  COMPLETED 

a.  The  VAA  Phase  11  study  directive  stated  that  the  primary  purpose  of  the  study 
was  to  implement  the  VAA  methodology.  The  VAA  Phase  El  Study  in  fact  refined  the 
methodology  as  developed  in  the  Phase  I  Study  and  built  tools  to  implement  the 
method.  In  addition  to  building  these  tools,  the  team  also  completed  the  work  and 
analysis  necessary  to  produce  VAA  Coefficients  for  41  systems.  This  might  have 
been  considered  the  end  of  the  study  and  that  all  objectives  had  been  met.  However, 
it  was  clear  to  the  team  that  this  did  not  constitute  analysis  or  implementation  of  the 
method  as  originally  intended  (as  a  tradeoff  tool  to  conduct  cost/benefit  analysis). 
Even  in  the  development  of  the  study  plan  for  the  VAA  Phase  11  Study,  there  was  a 
block  which  was  called  POM  support.  The  intent  of  the  team  was  to  assist  the  sponsor 
in  conducting  some  type  of  Value  Added  Analysis  in  support  of  the  POM.  This 
opportunity  came  in  the  form  of  the  LRRDAP  Review.  The  team  was  asked  to  assist 
in  the  review  of  the  LRRDAP  by  conducting  VAA  tradeoff  analysis  in  order  to  fit  the 
plan  into  the  current  OSD  fiscal  (POM)  guidance.  The  team  did  conduct  a 
preliminary  analysis  which  was  presented  to  the  DA  Council  of  Colonels,  which 
consisted  of  representatives  from  both  the  study  sponsors  and  the  Office  of  the 
Assistant  Secretary  of  the  Army  for  Research,  Development,  and  Acquisition 
(ASA(RDA)).  This  work  was  expanded  by  the  Council  into  a  series  of  QRAs  which 
culminated  in  a  briefing  to  the  (]hief  Staff  of  the  Army  (CSA)  Offsite  Review  for  the 
LRRDAP.  Table  9-1  provides  a  list  of  the  QRAs  conducted  in  support  of  the  CSA 
Offsite.  Two  additional  QRAs  were  conducted  in  support  of  the  Army  Select 
Committee  (SELCOM)  and  the  Secretary  of  the  Army.  However,  for  the  purpose  of 
demonstrating  the  implementation  of  the  Value  Added  Methodology,  the  work 
completed  in  support  of  the  Chief  of  Staff  Offsite  appears  to  be  the  most  appropriate. 


Table  9-1.  Value  Added  Analysis  QRAs  in  Support  of  CSA 


QRA  tasking 

Sponsor 

CAA  document 
number 

Comparison  of  LRA.MRP  and  LRRDAP  2.0 

DAMO-FD/ASPAD 
PAED/  SARD-Rl 

CAA-MR-92-29 

LRRDAP  3.0  General  Office  Review 

DAMO-FD/ASPAD 
PAED/  SARD-Rl 

CAA-.MR-92-24 

Aviation  and  Modernization  Program  Baseline 

ASPADPAED 

CAA-MR-92-29 

LRRDAP  3. 1  Chief  of  Staff  Offsite 

DAMO-FD 

CAA-MR-92-25 

LRRDAP  3. 1  .Modified  Chief  of  StalT  Offsite 
( President's  State  of  the  L'nion  Adjustments) 

DA.MO-FD 

CAA-MR-92-25 
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b.  Although  five  QRAs  were  conducted  in  support  of  the  CSA  OfTsite  and 
represent  separate  studies,  the  Modified  Chief  of  Staff  OfTsite  QRA  was  chosen  for 
partial  enclosure  in  this  paragraph.  The  team  felt  that  providing  a  description  of  the 
analysis  and  some  examples  of  the  products  provided  would  be  useful  in  helping  the 
reader  in  understanding  the  implementation  of  VAA.  The  following  provides  some  of 
the  highlights  from  the  Chief  Staff  Offsite  QRA. 

(1)  Purpose.  The  purpose  of  the  CSA  Offsite  QRA  was  to  assist  the  ARSTAF  in 
allocating  resources  for  modernization  programs  by  assessing  the  relative 
contribution  ("value  added")  of  selected  major  item  systems  (VAA  systems)  from  the 
LRRDAP3.1  file. 

(2)  Scope.  Using  the  appropriate  baseline  dollar  resources  (TOA)  and  forces, 
the  analysis  examines  the  relative  worth  of  the  VAA  systems  and  provides  an 
optimal  mix  of  for  the  VAA  systems.  The  following  assumptions  and  limitations  were 
used  in  the  analysis: 

•  All  costs  and  TOA  are  in  current  dollars. 

•  Production  campaigns  are  fixed. 

•  Length  of  campaign  is  based  on  force  structure  requirements  and 
production  capability. 

•  Used  LRRDAP  3.1  campaigns  matching  as  closely  as  possible. 

•  Constrained  to  Force  Package  I  or  MSR,  whichever  is  higher. 
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•  Constrained  to  appropriate  percentage  of  TOA  based  on  the  ratio  of  dollars 
allocated  to  the  VAA  systems  in  the  LRRDAP  3.1  file.  Figure  9-2  compares  the 
various  TOA  levels  for  VAA  systems  over  the  previous  analyses. 


Percent 
of  TOA 


0.70  r- 


Fiscal  year 

Figure  9-2.  Comparison  of  Proportion  of  TOA  for  VAA  Systems 


•  ASM  program  costs  reflected  the  latest  program  restructuring. 

(3)  General  Observations.  The  following  observations  have  been  drawn  from 
the  series  of  analyses  that  have  been  conducted  to  support  the  Modernization  Review. 
They  have  consistently  appeared  in  the  observations  of  individual  analyses. 

•  RAH-66  Comanche  with  LONGBOW  is  too  expensive  for  procurement  at  the 
levels  of  funding  considered  in  these  analyses,  in  spite  of  a  large  contribution  to  force 
performance. 

•  Although  separate  procurement  was  an  option,  RAH-66  Comanche  was 
always  procured  with  the  LONGBOW,  when  it  was  forced  into  a  solution. 

•  The  following  systems  appeared  to  be  "fillers;"  that  is,  they  are  recommended 
only  when  there  is  enough  TOA  left  to  fund  them.  They  are  systems  that  are 
frequently  traded  off. 


••  xMLRS  SADARM. 

••  Nonline  of  sight-combined  arms  (NLOS-CA). 
••  Amored  recovery  vehicle  (ARV'). 


9-9 


CAA-SR-92-10 


•  Ml  A2  Abrams  main  battle  tank  and  Block  III  tank  have  never  been 
recommended. 

•  The  relatively  high  cost  of  both  common  chassis  RDTE  and  lead  ASM  system 
procurement  (AFAS)  force  the  ASM  program  out  of  the  set  of  systems 
recommended  by  VAA. 

Figure  9-3  shows  commonalities  and  differences  between  LRRDAP  3.1  and  VAA. 


Included  in  VAA 

Included  in  both 

Included  in  LRRDAP 

Advanced  Firefinder 

AAWS-M 

AFAS 

AH-64  Apache  w/  LONGBOW 

AGS 

Block  III  tank 

AMS-H 

Advanced  QLTCKFIX 

FARV-A 

ATACMS  II 

ARV 

FIFV 

155mm  LtWt  How 

Avenger 

LOSAT 

NLOS-CA 

FMTV 

RAH  ~  -manche 

STINGRAY 

GBCS-L/H 

R.  .H-o6  Comanche  w/LON'GBOW 

GBS 

SADAR.M 

JSTARS 

PLS 

SADARM IMLRS) 

Neither  VAA  nor  LRRDAP 

.. 

TOW2B 

recommended  these  systems; 

VOLCANO 

WAM 

ADATS 

CMV 

< 

M1A2 

Figure  9-3.  VAA  -  LRRDAP  3.1  Comparison 


(4)  Description  of  Modified  Baseline.  The  following  is  a  description  of  the 
modified  baseline  used  to  support  the  Chief  of  StafT  Offsite: 

•  Systems  were  allowed  to  buy  into  Force  Package  II. 

•  The  following  systems  were  not  considered: 

•  ATAS,  155mm  field  artillery  extended  range  (155  ER),  155mm  terminally 
guided  projectile  (155  TGP),  MLRS-TGW,  advanced  special  electronic  mission 
aircraft  (ADV  SEMA),  vehicle  magnetic  signal  duplicator  (VEMASID),  heavy 
assault  bridge  (HAB),  and  heavy  equipment  transporter  (HET). 
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..  •  The  following  systems  were  considered  in  the  excursions: 

•  Air  Defense  Antitank  System  (ADATS),  line  of  sight  antitank  (LOSAT),  and 
RAH-66  Comanche  with  and  without  LONGBOW. 

•  Each  level  of  TOA  was  evaluated  as  a  separate  "optimization." 

•  TOA  includes  a  3.2  percent  growth  for  inflation  in  the  EPP  years. 

•  Only  three  levels  ofTOA  were  considered— $10.7,  $11.2,  and  $11.7  Billion. 

•  ASM  systems  linked  to  AFAS: 

•  FARV-A  must  be  procured  with  AFAS. 

•  All  other  ASM  systems  were  restricted  from  procurement  unless  AFAS  is 
procured. 

•  Army  Tactical  Missile  System  Block  11  (ATACMS 11)  costs  and  quantities  were 
directly  obtained  from  LRRDAP  3.1  file. 

•  Armored  Gun  System  (AGS)  forced  to  be  procured. 

•  Advanced  Missile  System-Heavy  (AMS-H)  program  delayed  2  years. 

(5)  Modified  Baseline  Observations.  Figure  9-4  provides  a  typical  example 
of  the  output  from  a  VAA  QRA.  The  following  is  a  description  of  the  modified 
baseline  observations: 

•  FY  00  was  no  longer  the  bottleneck  year,  restricting  buys  in  all  three  TOA 
levels. 

•  No  ASM  systems  were  recommended. 

•  The  primary  tradeoff  was  between  NLOS-CA  at  $10.7  billion  level  and 
ATACMS  n  at  higher  TOA  levels. 

•  At  the  $10.7  billion  level,  ATACMS  II,  and  SAD  ARM  were  recommended  vice 
ARV. 

•  At  the  $11.2  billion  level,  the  family  of  medium  tactical  vehicles  2.5/5-ton 
(FMTV  2.5/5)  and  AH-64  Apache  with  LONGBOW  were  recommended. 

•  At  the  $11.7  billion  level,  FMTV  and  AH-64  Apache  with  LONGBOW,  and 
NLOS-CA  were  recommended. 


9-11 


9-12 


e  9-4.  Chief  of  Staff  Offsite  Modified  Baseline 


CAA-SR-92-10 


9.-5.  VAA  FUTURE  DEVELOPMENT 

a.  Methodology.  The  study  team  believes  the  following  methodological 
improvements  or  research  should  be  conducted  to  enhance  the  VAA  capability. 

(1)  There  should  be  follow-on  research  to  establish  a  hierarchy  of  MOE  to 
evaluate  a  broader  range  of  systems  in  the  RDA  accounts.  The  next  iteration  should 
include  all  of  the  Defense  Acquisition  Board  (DAB)  and  Active  Component  (AC) 
systems  plus  additional  systems  from  the  current  list  of  Army  acquisition  programs 
to  capture  at  least  65  percent  of  the  total  dollars  associated  with  the  RDA 
appropriation.  By  way  of  comparison,  the  VAA  Phase  II  systems  captured  from  39 
percent  to  57  percent  of  the  total  RDA  TO  A. 

(2)  An  effort  should  be  made  to  refine  the  current  models  and  techniques, 
especially  the  optimization  and  multiattribute  utility  model,  for  developing  the 
single  measure  of  benefit  should  be  undertaken. 

(3)  A  test  case  should  be  developed  to  determine  the  requirement  for  VAA 
capability  in  other  appropriations  such  as  OMA,  MPA,  or  MCA.  If  a  requirement  for 
expansion  exists,  then  work  should  be  conducted  to  identify  or  develop  appropriate 
models  for  evaluating  other  appropriations.  Also,  the  survey  process  should  be 
redesigned  to  incorporate  the  need  to  do  multiple  appropriations. 

(4)  Analysis  should  be  conducted  to  determine  if  a  requirement  for  an  across- 
appropriation  balancing  within  the  VAA  context  exists.  If  a  requirement  exists,  then 
the  following  tasks  should  be  completed: 

(a)  Articulate  the  current  process. 

(b)  Translate  the  current  process  into  the  VAA  context  (start  point). 

(c)  Identify  ways  of  "improving"  the  process  by  applying  analytical 
techniques. 

(d)  Frame  a  feasible  methodology. 

b.  Implementation.  The  study  team  believes  the  following  implementation 
issues  should  be  addressed  in  follow-on  work. 

(1)  Work  should  be  completed  to  determine  the  extent  to  which  VAA  capability 
needs  to  be  implemented  in  the  METAPHOR  environment. 

(2)  Another  possible  task  for  a  follow-on  study  is  to  determine  if  useful  data  for 
VAA  analysis  can  be  developed  using  the  METAPHOR  and  the  corporate  Army  data 
bases  currently  available. 

(3)  An  effort  should  be  made  to  implement  the  resource  allocation  module. 

(4)  To  successfully  implement  the  VAA  methodology,  there  is  a  need  to  create 
real  connectivity  for  all  users  of  VAA,  both  in  terms  of  hardware  and  software. 
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MEMORANDUM  FOR  DIRECTOR,  U.S.  ARMY  CONCEPTS  ANALYSIS  AGENCY. 

8120  WOODMONT  AVENUE,  BETHESDA,  MARYLAND 
20814-2797 

SUBJECT;  Army  Program  Value  Added  Analysis  -  Phase  II 
(VAA  94-99). 


1.  PURPOSE  OF  STUDY  DIRECTIVE:  This  directive  establishes 
objectives  and  provides  guidance  for  the  conduct  of  the  Value 
Added  Analysis  -  Phase  II  Study. 

2.  BACKGROUND:  Headquarters,  Department  of  the  Army  (HQDA) 
needs  analysis  to  support  the  development  of  a  balanced  and 
effective  Army  Program  within  Department  of  Defense  resource 
guidance.  The  Array  requires  a  methodology  for  identifying  and 
comparing  the  warfighting  contribution  of  major  item  systems 
proposed  for  research,  development  and  acquisition  during  the 
time  period  of  the  Program  Objective  Memorandum  (POM).  The 
Value  Added  Analysis  Methodology  developed  by  U.S.  Army 
Concepts  Analysis  Agency  (CAA)  during  the  Value  Added  Analysis 
-  Phase  I  Study  uses  a  hierarchical  assessment  framework  for 
assessing  return  on  investment  as  it  relates  to  explicit  and 
implicit  measures  of  benefit  when  compared  with  cost. 

3.  STUDY  SPONSORS  AND  SPONSOR’S  STUDY  DIRECTOR: 

a.  The  Director,  Program  Analysis  and  Evaluation  (DPAE), 
Office  of  the  Chief  of  Staff  Array,  is  a  co-sponsor. 

b.  The  Assistant  Deputy  Chief  of  Staff  for  Operations  and 
Plans,  Force  Development  (ADCSOPS-FD)  is  a  co-sponsor. 

c.  The  PA4E  Study  Sponsor’s  Representative  is  LTC  James  N. 
Rlchmann,  (DACS-DPZ).  The  DAMO-FD  Study  Sponsor's 
Representative  is  COL  Roger  A.  Brown.  (DAMO-FD). 

4.  STUDY  AGENCY:  U.S.  Army  Concepts  Analysis  Agency. 

5.  TERMS  OF  REFERENCE: 

a.  Purpose.  The  purpose  of  this  study  is  to  assist  the 
Army  Staff  (ARSTAF)  in  allocating  resources  for  modernization 
programs  by  assessing  the  relative  contribution  ("value  added") 
of  major  item  systems  toward  the  capability  of  US  Army  forces. 
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b.  Definitions. 

(1)  Value  Added  -  refers  to  the  marginal  return  on 
investment  as  measured  using  explicit  effectiveness  values  and 
Implicit  effectiveness  values  as  compared  to  cost. 

(2)  Major  Item  System  -  is  a  combination  of  major 
items, separately  authorized  secondary  items,  component  major 
items,  associated  items  of  equipment,  spare  and  repair  parts, 
munitions,  personnel  and  facilities  that  are  jointly  used  to 
accomplish  a  specific  function. 

c.  Scope. 

Cl)  Baseline  program  is  President’s  Budget  FT  1992  and 
U.S.  Program  Force  (FT  1995). 

(2)  The  analysis  will  examine  the  ROA  appropriations  and 
other  related  appropriations  as  needed  to  investigate  the  full 
programing  of  selected  major  item  systems. 

(3)  About  50  Major  Item  Systems  (see  enclosure  1)  based 
on  the  following  selection  criteria:  systems  without  a  Mission 
Needs  Statement  (MNS)  or  other  requirements  document  (such  as  a 
Required  Operational  Capability  (ROC))  as  of  1  October  1991  will 
not  be  Included,  systems  with  large  dollar  amounts  in  current 
and  proposed  programs  will  be  included,  and  finally  inclusion  of 
a  system  is  subject  to  our  ability  to  execute  in  terms  of  data 
and  modeling  availability. 

(4)  Scenario  Conditions:  NATO  Central  Europe,  Post-CFE, 
Defense;  SWA  Defense  with  Counter  Attack;  NEA  Counter-Offensive; 
Consistent  with  TRADOC  acenario(s). 

(5)  D.S.  Army  conventional  operations  capabilities  will 
be  assessed  in  terms  of  force  packages,  major  item  systems 
programmed  for  acquisition,  and  anticipated  threat  as  of  the  end 
of  FT  1996,  FY  2001,  and  FY  200«. 

(6)  Conflict  Type;  Conventional 

d.  Objectives. 

(1)  To  determine  VAA  coefficients  and  feasible 
acquisition  alternatives  for  major  item  systems  proposed  by  the 
Headquarters,  Department  of  the  Array  Long  Range  Research 
Development  Acquisition  Plan  (HQDA  LRRDAP)  and  constrained  by 
Modernization  Total  Obligation  Authority  (TOA).  The  process 
must  measure  and  analyze  the  capability  of  US  Army  farces  to 


2 


CAA-SR-92-10 


DACS-DPZ 

SUBJECT;  Array  Program  Value  Added  Analysis  -  Phase  II 
(VAA  94-99). 


conduct  conventional  operations  in  scenarios  consistent  with  the 
Illustrative  Planning  Scenarios  of  the  Defense  Planning  Guidance 
FT  1994-1999. 

(2)  To  identify  and  develop  a  Value  Added  Analysis 
Capability  (VAAC)  to  include  all  appropriate  hardware,  software, 
and  interfaces.  The  VAAC  must  tap  major  authoritative  Array  data 
bases  such  as  the  Total  Array  Equipment  Distribution  Program 
(TAEDP),  Force  Accounting  System  (FAS),  and  The  Army  Force  Cost 
System  (TAFCS). 

(3)  To  identify  or  develop  models  and  techniques  that 
support  the  VAA  Methodology.  The  VAAC  and  related  models  must 
be  capable  of  operating  in  a  "quick  turnaround"  environment, 
defined  as  1  week  or  less. 

(4)  To  conduct  a  demonstration  of  the  refined  methodology 
and  VAAC  prior  to  the  building  of  the  1994-1999  POM. 

(5)  To  continue  the  refinement  and  implementation  of  the 
VAA  Methodology  for  estimating  the  value  added  of  either 
competing  major  item  systems  or  Management  Decision  Packages 
(MDEPs)  to  the  Total  Army  Program. 

e.  Timeframes. 

(1)  Fiscal  obligation  analysis  timeframe:  FY  1994, 

FY  1999,  and  FY  2006. 

(2)  Fielding  analysis  timeframe:  FY  1996,  FY  2001,  and 
FY  2008. 

(3)  Combat  analysis  timeframe;  Blue  forces  FY  1995  and 
Red  forces  FY  1996,  FY  2001,  and  FY  2004  (Blue  force  is  equal  to 
FY  1995  basecase  year  which  represent  forces  on  hand  as  the 
result  of  the  President's  FY  1992  budget). 

f.  Limitations.  The  VAAC  will  initially  be  installed  at  CAA 
because  the  hardware  and  software  required  is  currently 
available  only  at  CAA. 

g.  Assumptions. 

(1)  The  President's  FY  1992  Budget  position  will  be  used 
for  both  the  Program  Optimization  and  Budget  Evaluation  (PROBE) 
and  Research  Development  and  Acquisition  Information  System 
Agency  (RDAISA)  databases. 
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(2)  The  data  survey  results  are  assumed  to  reflect 
decision  maker  positions  and  provide  a  means  of  modeling  the 
effect  of  the  senior  Army  leadership  on  the  Army  decision  making 
process. 

(3)  The  measure  of  effectiveness  (MOE)  selected  measures 
the  utility  of  a  major  item  system. 

(4)  There  is  a  two  year  lag  from  expenditures  of 
procurement  dollars  to  item  fielding. 

h.  Essential  Elements  of  Analysis  (EEA). 

(1)  What  is  the  "value  added”  of  a  select  number  of  major 
item  systems  to  be  considered  in  the  1994-1999  POM? 

(2)  What  kind  of  characteristics  shoi'ld  the  VAAC  have  in 
order  to  implement  the  VAA  Methodology? 

(3)  What  kinds  of  tools,  techniques  and  models  are  needed 
to  support  the  VAA  Methodology? 

(4)  What  explicit  effectiveness  models  are  most 
appropriate  for  a  Value  Added  Analysis  Study  (Value  Added 
Analysis  1994-1999)? 

(5)  What  refinements  are  needed  to  improve  the  VAA 
Methodology? 

6.  RESPONSIBILITIES. 

a.  D,PA&E  will: 

(1)  Provide  a  study  sponsor  representative  and  study 
guidance. 

(2)  Provide  a  study  team  member. 

(3)  Provide  support  to  the  analysis  for  data  required  to 
conduct  the  VAA  study. 

b.  ADCSOPS-FD  will: 

(1)  Provide  a  study  sponsor  representative  and  study 
guidance. 

(2)  Provide  a  study  team  member. 
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(3)  Provide  support  to  the  analysis  for  data  required  to 
conduct  the  VAA  study. 

c.  CAA  will: 

(1)  Conduct  the  Value  Added  Analysis  Study. 

(2)  Designate  a  study  director  and  provide  the  study 

team. 

(3)  Coordinate/communicate  with  the  PAED.  ODCSOPS.  TRAC, 
CEAC,  ASARDA,  AMC  and  others  for  data  and  information  to 
accomplish  the  study. 

(4)  Provide  periodic  in-process  reviews  (IPRs)  as 
requested  by  the  study  sponsors. 

7.  REFERENCE.  The  draft  Value  Added  Phase  I  Report  provides 
the  basis  for- the  methodology  and  approach  to  Phase  II.  Other 
references  include  President's  FY  1991  Budget  and  CAA's  Army 
Resource  Integration  and  Management  Study. 

8.  ADMINISTRATION! 

a.  Support. 

(1)  Funds  for  travel/per  diem  will  be  provided  by  the 
parent  organizations  of  the  individuals  traveling. 

(2)  ADP  support  will  be  provided  by  CAA. 

(3)  Secretarial  support,  reports,  and  publication  will  be 
provided  by  CAA. 

(U)  ODCSOPS  will  provide  access  to  and  use  of  the 
METAPHOR  Computer. 

b.  Milestone  Schedule. 

(1)  Key  milestone  dates. 

Initial  SAG 

Refinement  of  Methodology  Completed 
SAG  Review  of  Weights  and  SIAM  Factors 
VAAC  Complete  /  Software  Testing  Begins 
Analyze  Results  of  Demonstration 
POM  Support  Prep  Complete 
Support  POM  Development 


-H-  June-  199 1 
16  July  1991 
1  July  1991 
25  July  1991 
12  August  1991 
1  October  1991 
November  1991 
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Final  SAG  28  Februarv  '992 

VAA  Phase  II  Ends  27  March  1992 

(2)  A  complete  schedule  is  at  enclosure  2. 


c.  Control  Procedure.  The  study  sponsors'  representatives 
will  coordinate/communicate  within  the  HQDA  staff  and  between 
the  ARSTAF  and  the  study  agency. 


2  Encls^  JEROME  H.  GRANRUD 
Major  General,  USA 
Assistant  Deputy  Chief 
of  Staff  for  Operations 
and  Plan,  Force 
Development 


THOMAS  P.  CARNEY 
Major  General,  US^ 
Director,  Progran 

Analysis  and  E\  !uation 


6 


B-6 


CAA-SR-92-10 


SHOBT  NAME 

NOMENCLATURE 

CAA 

.1  AH-1  G/S 

Csbrt  Attack  Halo 

SC 

i  AH-iA 

AgicK* 

BC 

3  ATACM6  I 

Army  Tactical  Miaiil*  Syi  Blk  I 

BC 

A  BFV 

Bradlty  Fighting  Vahlcla 

BC 

3  CHAPPARRAL 

Chapparral 

BC 

9  FAASV 

FA  Ammo  Sup'ply  Vthiela 

BC 

7  GR/CS 

Gu a r d r a  1 1 /Comm  Snir-Corps 

BC 

8  HAWK 

HAWK 

BC 

9  HTARS 

KEMTT  Tankar  Aviation  Rtlutling 

BC 

10  Ml 

Abrama  Tank  103mm 

BC 

11  Mt09 

1  3  5mm  Hewi m t  SP 

BC 

12  Ml  10A2  S'* 

3"  Howl  1 itr 

BC 

13  Mils  lOSmm  h. 

Ml  to  tOSmm  How 

BC 

14  Miea 

Vulcan  AOA  Gun 

BC 

IS  MIAl 

Abrams  Tank  120mm 

BC 

18  M9  ACE 

M9  Armortd  Combit  Eirthmovtr 

SC 

17  M'RS 

Mulitpla  Launch  Rockat  Syjtam 

ac 

1  8  OBV 

Obstacla  Sraaching  Syi  <Mll 

BC 

IP  OH-39D 

Army  Halo  Imp  Pgm 

BC 

20  STINGER 

STINGER 

BC 

21  NET 

Haavy  Equipir.ant  Transport 

3C/  1 

22  SAOARM  MLRS 

Sanaa  t  Daatroy  Armor  MLRS 

BCi  1 

23  TOW  :A/28 

TOW  II  AT  Msl 

BC  /  1 

24  AAWS-M 

Adv  Anti«Armor  Wpn  Sys-Mad 

■ 

23  ADATS 

Ulnt  of  Fwd .  Ht4vy 

1 

28  AEI 

Armanant  Enhancamant  I  n i  l  a 1 1 va i Ammo ) 

1 

27  AFA8 

Adv  FA  Systam  (ASM) 

1 

2  8  AOS 

Armorad  Gun  Svstam 

. 

29  AH-84  LB 

AH-84C  Longbow  Apaeha 

30  AMS-H 

Adv  Ml  SSI  la  Sys-Heavy 

1 

31  ATACM6  II 

Army  Tactical  Misslla  Syi  Blk  11 

1 

32  ATAS 

Alr-to-Al.r  STINGER 

1 

33  AVENGER  AGS 

Los  Raar/Pad  Mtd  STINGER 

1 

34  Adv  Ol/ICKFIX 

Adv  OU 1 CKF 1 X / Aa r I  a  1  Com  Snsr  Oiv 

1 

3S  BLOCS  III 

Block  III  Ms  in  Ba  1 1 1  a  Tank 

1 

38  CMV 

Combat  Mobility  Vah  Naxi  Oantraticn 

I 

37  FIFV 

Future  Infantry  Fighting  Vahicla 

1 

38  HIP  (MIOOAS) 

Howl  tsar  Improvamant  Program 

1 

39  JSTARS 

Joint  Surv  and  Attack  Radar  System 

1 

40  LH  w/LONOBOW 

Light  Hflo  w/Lor. gbow 

I 

4  1  LH 

Light  Helicopter 

, 

42  LOSAT 

Line  of  Sight  Anti-Tank 

1 

43  LI  Wgt  135  How 

Light  Weight  135  Howitsar 

1 

44  MIA2 

Tank 

1 

43  MLRS  TOW 

MLRS  Tarminallv  Guided  Warhead 

1 
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short  name 

NOMENCLATURE 

CAA 

46  NLOS-AT 

Non  Lint  el  Sight  -Anti-Tink 

1 

47  PATRIOT 

PATRIOT  Syitim  Orowih 

( 

4  8  WAM 

Widl  Arii  Mine 

; 

49  IS5  FAEB 

153  FAER  (EXTENDED  RANGE' 

Z 

SO  ISS  TOP 

133  Tirminilly  Ouidid  Projectlli 

z 

31  AH-64  Imp 

Apichi  Improviminti 

2 

32  ARV 

Armored  Ricowiry-  Vinieii 

2 

33  Adv  SEMA 

Adv  Spiciil  Elec  Min  A/C 

2 

34  Ad  Vinci  Plrilindir 

Ad  Vince  Firilindir 

33  Air  VOLCANO 

Subiyilim  el  VOLCANO  lor  'JH-fO 

2 

36  CORPSAM 

CcTpi  Sur  i  icf  1 0*  Al  r  Mil  Syt 

2 

37  FAHV-A 

Futuri  Armd  Rliupply  Vth  (ASM) 

2 

38  FLTV 

Fimily  o(  Light  Tic  Vth  ihumVEE) 

2 

39  FMTV 

Fimtly  o(  Mtd  Tie  Vth  (5-Ton  TRX! 

2 

60  Flriflndtr  Slock  II 

Firtfindir  aiock  it 

2 

61  OBCS-L/H 

Ond  Sited  Common  Sensor  (LlGHT/HVyi 

2 

62  OBS 

Ground  Stsed  Sensor 

2 

63  OLTR 

Ond  Lchd  Tioit  Rtinbow 

2 

64  HAS 

Heivy  Asiiult  Sridge  (Ml  AVLB) 

2 

63  MPIM 

Multipurpose  Ind  Munition  (AT  Wpn ) 

2 

66  PLS 

Piltitised  Loiding  Syt 

2 

67  3TINORAY 

Combit  Protection  Device 

2 

66  VEMARIO 

Vehicle  Mignetle  Slg  Duplicitor 

2 

69  VOLCANO 

Multiple  Mine  Deliver  Syi 

2 

70  ADOS 

Army  Dili  Distribution  Sys 

3 

71  AFATDS 

Adv  FA  Ticticil  Dili  Svi 

3 

72  AOSM 

Army  Grnd  Stition/Comm  Gnd  Jvittr. 

3 

7  3  AMV 

Armored  Miint  Veh 

3 

74  APOB3 

Antlpers  Obs  Breiching  System 

3 

73  ABAS 

All  Source  Antlyils  Set 

3 

76  ATCSS  Common  H/S 

ATCSS  Common  H i r dwt r e / S o ( i wt r e 

3 

77  ATLAS 

All  Ttrriin  Lit*.  Articulitcd  Svt 

3 

78  CCTT  (NSTO) 

Close  Combit  Ticticil  'rimer 

3 

79  CSSC3 

Cbt  Ser  Spt  Control  Sys 

3 

80  CTT 

Cmd-r'i  Ticttcil  Term/CTT  Hybrid 

3 

81  FAAO  C21 

Fwd  Air  Oe<  Commend  ind  Control 

3 

82  FAAPC 

FA  Ammo  Proeesting  Center 

3 

33  FAAV 

Future  Atttek  Aeriil  Vehiclt 

3 

8  4  FED 

Forwitd  Entry  Device 

3 

83  FRV 

Future  Recovery  Vehlole 

3 

86  FSV 

Fire  Suppo  r  t  Vehicle 

3 

87  HAWK  Swiipdown 

HAWK  Sweepdown 

3 

88  HIMAO  Ridir 

H 1 gh / I n t t rmtd  Air  Dtltnie  Fidir 

3 

39  HIMARS 

High  Mob  Arty  Skt  Syi 

3 

90  IFTE 

Inltgriitd  Fimily  of  Tost  Eq 

i 
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SHORT  NAME 


NOMENCUATURE 


CAA 


01  IHFR  Upgradt 

Imp  HF  Radio  Upgradt 

3 

92  IMETS 

Inttgrattd  Mat  Syi  (Waathtr) 

3 

43  LAMP-H 

Llghitr.  Amphibian  Htavy  Lill  60T 

3 

94  LH/P3I 

Light  Htlo't  Product  Improvtmtnti 

3 

93  M109A3  MOD 

M109A3  MOD 

3 

96  M198  MOD 

tSSnun  HowMmp  Armamint  Sys 

3 

9  7  MCS 

Manutvtr  Control  Syatem 

3 

9S  MLAW 

Manportablt  Lattr  Aaiaull  Wtapon 

3 

99  M8E  Enhtnetffltntt 

Mobtlt  Subt  Eq  Enhanetmtnti 

3 

100  MTV>LE 

Mod  Tact  Vth  Lift  Exttnilon 

3 

101  Mint  Ot 1 te 1 1  on 

Ot  t  te  1 1  on  St  t ,  Mint 

3 

102  Hultl>Optlan  Fuit 

Multl'Option  Futt  Artllltry 

3 

103  NBC  Otitel 

NBC  fftttct,  ID.  4  Warning 

3 

104  PATRIOT  Swttpdown 

PATRIOT  Swttpdown 

3 

103  RCVR  Ttoh 

Coiiunun  1  ea  1 1  ont  ECM  AO/ED 

3 

106  SINCOARS 

Combat  Ntt  Radioa 

3 

107  STAMIOS 


Sl*nd«((  Min*(t*ld  Dtlietien  Sy< 


106 

109 

no 

Soldltr  Syattm 

TENCAP-ELECT 

TENCAP-IMIT 

Clothing 

Tactical 

Tactical 

and  Individual  Equipmani 

Ex  on  Natl'  Capa b  1  1  1  n t e a - E 

Ex  on  Natl'  Capabi 1 V 1 1 1 > a  - 1 

3 

3 

3 

1 1 1 

ATMO 

Ac  1 1  vt  Tactical 

Mai 

Ot( tnat 

4 

1  12 

UAV.CR 

Unmanned 

Atrial 

Vth 

Clott  Rg 

4 

1 1  3 

UAV-Endurt 

Unmanned 

Atrial 

Vth 

Endurance 

4 

1  1  4 

UAV-SR 

Unmanned 

Atrial 

Vth 

Short  Rg 

4 

CRITERION: 

1  THE  SYSTEM  HAS  A  REQUIREMENTS  DOCUMENT  HIGHER  THAN  AN  OtO.  HAS  A  LARGE  Hi 

HAS  GIVEN  THE  ITEM  AS  A  "1"  PR  I ,  AND  HAS  AVAILABLE  DATA  THAT  CAN  BE  MOI 

2  SYSTEM  IS  AN  ISSUE  AND  MAY  BE  ABLE  TO  BE  MODELED 

3  THE  SYSTEM  HAS  A  LOW  PRIORITY  FROM  OOCSOPS  (  PALE,  HAS  A  LOW  DOLLAR  AMOUl 

4  SYSTEMS  THAT  WILL  NO  LONGER  BE  CONSIDERED 


BASE  SYSTEM  OB  IOC  SS . 

I .  FIELDED  AS  OF  1093 

2  NOT  AN  ISSUE  FOR  ADDITIONAL  FUNDING 
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APPENDIX  D 

DECISIONMAKER  SURVEY  MEMORANDUMS 


Presented  in  this  appendix  are  the  memorandums  identifying  decisionmaker 
surveyees  and  providing  information  regarding  background  and  conduct  of  the 
survey. 
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1  August  1991 

CSCA-FSR 

MEMORANDUM  THRU  Mr.  Steve  Siegel,  FSR 

COL  Harrington,  FS 

FOR  Director,  Concepts  Analysis  Agency 

SUBJECT:  Identification  of  VAA  Decision  Maker  Surveyees 

1.  Reference: 


a.  FONECON,  LTC  Richmann,  DACS-DP;  LTC  Koury,  CSCA-FSR;  and 
MAJ  Clayton,  CSCA-FSR  at  221300JUL91 . 

b.  Meeting,  29JUL91,  LTC  Richmann,  DACS-DP;  LTC  Pollster,  DAMO- 
FDR;  and  LTC  Koury,  CSCA-FSR,  at  the  Pentagon. 

2.  The  following  individuals  were  tentatively  identified  as  target  surveyees  for  the 
conduct  of  Value  Added  Analysis  (VAA)  Phase  II: 

ASA(RDA) 

DUSA(OR) 

Deputy  ASA  (Plans  and  Programs) 

Military  Deputy  to  the  ASA(RDA) 

Comptroller  of  the  Army 

DCSOPS 

DCSLOG 

DISC4 

DAS 

DCSPER 

DCSINT 

Director,  PAED 

ADCSOPS  (Force  Development) 

Director,  CAA 
Chief,  Army  Reserve 
Chief,  National  Guard  Bureau 
Chief  of  Engineers 
DCSRM,  FORSCOM 
DSCRM,  TRADOC 
DCSCD,  TRADOC 
DCSRM,  AMC 

DCS(Development,  Engineering,  and  Acquisition),  AMC 
DCSRM,  Health  Services  Command 
DCSRM,  USAREUR 
DCSRM,  WESTCOM 


3.  VAA  sponsors  have  initially  approved  the  list  above.  Another  list  of  subject 
matter  experts  for  the  scoring  of  the  secondary  [factors  will  be  provided  at  a  future 
date. 
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4.  POC,  this  action, .MAJ  Clayton,  CSCA-FSR,  (301)295-5291/0211. 


CF: 


ROBERT  R.  KOURY 
LTC,  IN 
Study  Director 

LTC  Richmann,  DACS-DP 
LTC  Pollster,  DAMO-FDR 
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CSCA-FSR  22  August  1991 


MEMORANDUM  FOR  VAA  DECISION  MAKER  SURVEYEES 


SUBJECT:  VAA  DECISION  MAKER  SURVEY  READ-AHEAD  PACKAGE 

1.  Purpose.  This  memo  will  provide  information  on  the  background  and  conduct  of 
the  Value  Added  Analysis  (VAA)  Decision  Maker  Briefing  and  Survey 

2.  Problem.  The  Army  requires  the  capability  to  systematically  &  efficiently 
evaluate  the  relative  contribution  ("Value  Added")  of  Major  Item  Systems  toward  the 
capability  of  US  Army  forces. 

3.  General.  Value  Added  Analysis  is  the  self-refining  procedure  developed  to  provide 
a  value  of  relative  contribution  for  systems  under  consideration  in  the  upcoming 
POM.  Divided  into  effectiveness  and  cost  arenas,  VAA  uses  existing  operations 
research  and  management  techniques.  It  combines  the  macro  perspective  input  from 
decision  makers,  to  derive  weighted  criteria,  with  the  system/p^rogram  specific  input 
of  technical  experts,  simulation  results,  and  cost  estimations.  In  combination,  the 
information  and  algorithms  are  put  into  an  automated  environment  where  real-time 
evaluation  is  made  available  to  those  integral  to  POM  development. 

4.  Chronology.  The  current  phase  of  VAA  (II)  was  initiated  in  May  1991  with  a 
signed  study  directive.  Surveys  are  to  be  completed  by  mid-September.  Final 
demonstration,  in  line  with  POM  94  99  development,  is  to  be  completed  by  OCT  91. 

5.  Surveys.  The  startpoint  and  direction  of  the  efTort  lies  with  the  input  of  decision 
makers  who  are  polled,  via  pairwise  comparison  techniques,  on  respective  weights  of 
timeframes-scenarios,  implicit  (unquantifiable)  factors  affecting  programs,  and 
measures  of  effectiveness.  Additionally,  a  system  by  system  comparison  is  done  in  a 
survey  of  technical  experts  to  determine  the  relative  potential  of  each  system  in  each 
of  the  implicit  factors.  All  surveys  are  non-attributable  and  information  is  handled 
as  close  hold  by  the  VAA  team. 

6.  CUonductof  Decision  Maker  Surveys.  Desired  to  be  completed  in  a  30  minute 
period,  the  session  begins  with  a  briefing  of  VAA,  followed  by  an  example,  and  then 
with  the  actual  surveys.  Each  of  the  four  surveys  is  introduced  by  purpose,  necessary 
definitions,  and  conventions.  The  survey  instruments  themselves  consists  of  index 
card  sized  questionnaires  with  a  single  response  required  on  each.  The  response  is  a 
mark  on  a  scale  with  respect  to  the  two  items  being  compared.  There  are  a  total  of  53 
comparisons.  Finally,  a  period  is  provided  for  surveyees  to  offer  their  insights  in 
response  to  the  surveys  and  the  VAA  process.  Although  not  inherent  to  the  survey, 
this  feedback  is  of  paramount  importance  and  will  be  used  in  refining  the  VAA 
process  in  the  future. 

7.  POC,  this  activity,  MAJ  Robert  Clayton,  CSCA-FSR,  (301)295-021 1/5291 . 


ROBERT  KOURY 
LTC,  IN 
Study  Director 
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APPENDIX  E 
COST  DOCUMENTATION 


E-1.  INTRODUCTION.  This  appendix  provides  additional  documentation  to 
support  the  VAA  cost  analysis  process  described  in  Chapter  7.  This  appendix 
includes  an  example  of  the  LCCM  template,  definitions  of  the  five  major  cost 
categories,  and  an  example  of  the  LCCMl  and  LCCM2  spreadsheets. 

E-2.  LCCM  TEM  PLATE.  Figure  E-1  provides  a  complete  template  of  the  P-92  cost 
codes  included  in  the  LCCM.  The  column  titled  "APPN"  shows  the  appropriation 
coding  corresponding  to  the  P-92  code.  Similarly,  the  column  titled  "materiel  cost 
category"  provides  the  coding  for  weapon  system  cost  categories  (e.g.,  flyaway  cost) 
used  to  calculate  the  five  major  weapon  system  cost  categories  displayed  in  Chapter 
7,  Table  7-5. 
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E-.3.  "BIG  5"  COST  CATEGORIES.  The  five  major  cost  categories,  commonly 
referred  to  as  the  "Big  5,"  are  described  in  this  paragraph. 

a.  Development.  All  RDTE  costs  associated  with  the  development  of  th<^ 
materiel  system.  These  include  development  costs  for  system  arman^''nL,  u  aining 
devices,  ammunition,  missiles,  and  modifications  and  integrated  logistic  support 
requirements. 

b.  Production.  A  sum  of  all  procurement  appropriation  funded  custs  resulting 
from  the  production  and  introduction  of  the  materiel  system  into  the  Army's 
operational  inventory.  These  include  the  following; 

(1)  All  costs  to  the  government,  defined  as  contractor  costs  plus  in-house  costs 
necessary  to  transform  the  results  of  development  into  a  fully  operational  system 
necessary  to  initiate  operations. 

(2)  Procurement  costs  of  both  those  recurring  in  nature,  i.e.,  costs  which  occur 
repeatedly  during  production  and  delivery  to  user  organizations  and  those 
nonrecurring  in  nature,  i.e.,  costs  which  are  required  to  establish  a  production 
capability. 

(3)  All  costs  resulting  from  production  and  introduction  into  inventory, 
irrespective  of  the  allocation  method.  These  include  unit  equipment,  maintenance 
float,  and  training  usage  classification. 

c.  Military  Construction.  Includes  both  system  specific  and  nonsystem  specific 
requirements.  System  specific  implies  the  materiel  system  cannot  be  fielded  without 
construction.  Examples  of  system  specific  construction  projects  are  simulator 
buildings,  missile  bunkers,  and  construction  of  billets  associated  with  fielding  new 
organizations  for  new  systems. 

d.  Fielding.  The  process  of  introducing  a  new  materiel  system  and 
simultaneously  redistributing  the  replaced  materiel  system  to  a  unit  or  a 
prepositioned  site  with  sufficient  resources  to  achieve  the  user's  given  mission 
objectives.  Fielding  begins  when  the  ownership  of  the  system  is  passed  from 
manufacturer  to  the  government,  and  ends  when  the  materiel  system  is  incorporated 
into  a  table(s)  of  organization  and  equipment  (TOE)  or  table  of  distribution  and 
allowances  (TDA)  unit  and  operates  to  permit  the  unit  to  accomplish  its  mission. 

e.  Sustainment.  The  continuing  recurring  process  of  operating  and  maintaining 
force  structure  and  materiel  systems  to  perform  assigned  tests  and  missions.  The 
level  of  sustainment  is  a  function  of  force  allocation  and  training  objectives  as  well  as 
the  operating  tempo  assigned  to  the  individual  materiel  systems.  Sustainment 
generally  begins  when  the  system  is  fielded  and  ends  when  it  leaves  the  Army 
inventory. 
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E:4.  LCCMl  and  LCCM2.  Figures  E-2  and  E-3,  respectively,  show  the  LCCMl  and 
LCCM2  data  for  production  and  sustainment  costs.  The  top  half  of  Figure  E-2 
illustrates  the  LCCMl  for  the  quantity  shown.  The  lower  half  of  Figure  E-2  (LCCM2) 
provides  the  revised  production  costs  that  are  associated  with  the  revised  quantities 
shown,  based  on  proportional  relationships  calculated  from  LCCMl  (see  Chapter  7). 
Similarly,  changes  in  sustainment  costs  are  shown  in  Figure  E-3. 
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Figure  E-2.  LCCMl  and  LCCM2  (production  costs) 
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Figure  E-3.  LCCMl  and  LCCIV12  (sustainment  costs) 
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APPENDIX  G 

SPONSOR’S  COMMENTS 

STUDY  CRITIQUE 


(This  document  may  be  modified  to  add  more  space  for  responses  to 
questions.) 


1.  Are  there  any  editorial  comments?  N d _  If  so,  please  list 

on  a  separate  page  and  attach  to  the  critique  sheet. 


2.  Identify  any  key  issues  planned  for  analysis  that  are  not 
adequately  addressed  in  the  report.  Indicate  the  scope  of  the 


additional  analysis  needed. 


3.  How  can  the  methodology  used  to  conduct  the  study  be  improved? 


4.  What  additional  information  should  be  included  in  the  study  report 
to  more  clearly  demonstrate  the  bases  for  the  study  findings? 


5.  How  can  the  study  findings  be  better  presented  to  support  the  needs 
of  both  action  officers  and  decisionmakers?  _ er 


6,  How  can  the  written  material  in  the  report  be  improved  in  terms  of 
clarity  of  presentation,  completeness,  and  style?  r- _ 
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STUDY  CRITIQUE  (continued) 


7.  How  can  figures  and  tables  in  the  report  be  made  more  clear  and 
helpful?  yu  _ _ 


8.  In  what  way  does  the  report  satis^  the  expectations  that  were 
present  when  the  work  was  directed?  ^ 

In  what  ways  does  the  report  fail  to  satisfy  the  expectations? 

/t/j/yuc — _ ; _ ^ _ ; _ 


9.  How  will  the  findings  in  this  report  be  helpful  to^e  organizatjon 
which  directed  that  the  work  be  done?  Hruuu 


nzatio 


I  r^i/y*u.£u^ 


If 


'ey  will  not  be  helpful,  please  explain  why  not. 


10.  Judged  overall,  how  do  you  rate  the  study?  (circle  one) 


Poor 


Fair  Average 


/  , 


Good 


Excellent 
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DEPARTMENT  OF  THE  ARMY 

OFFICE  OF  THE  DEPUTY  CHIEF  OF  STAFF  FOR  OPERATIONS  AND  PLANS 
WASHINGTON,  DC  20310-0400 


mptir  TO 
attintiOn  0* 


DAMO-FDZ  12  August  1992 

MEMORANDUM  FOR  DIRECTOR,  US  ARMY  CONCEPTS  ANALYSIS  AGENCY 

SUBJECT:  Army  Progr2im  Value  Added  Analysis  94-99  (VAA  94-99) 
Study  Report  CAA-SR-92-10 


1.  Reference  Draft  study,  dated  June  1992,  SAB. 

2.  The  draft  study  report  has  been  evaluated  and  accurately 
documents  the  VAA  methodology  and  work  performed  to  support  the 
Long  Range  Research  Development  and  Acquisition  Plan  and  the  POM 
for  FY  1994-1999.  The  methodology  and  its  application  provided 
significant  insight  to  tradeoffs  in  acquisition  of  major 
modernization  systems  and  their  war  fighting  value.  In 
summary,  the  development  of  the  methodology  was  well  formulated 
and  thorough,  the  determination  of  the  "values'*  for  selected 
systems  was  sound,  and  the  presentation  of  results  was  timely, 
clear  and  succinct.  The  study,  its  director  and  supporting 
analysts,  provided  an  exceptional  contribution  to  decision 
makers  leading  to  a  balanced  FY  1994-99  POM. 


3.  A  study  critique  is  attached.  No  additions  or  deletions  to 
the  distribution  list  are  suggested. 


Enel 


JAY  M.  GARNER 
Major  General,  GS 
Assistant  Deputy  Chief  of  Staff 
for  Operations  and  Plans, 
Force  Development 
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STUDY  CRITIQUE 


(This  document  may  be  modified  to  add  more  space  for  responses  to 
questions.) 

1.  Are  there  any  editorial  comments?  Mo  If  so,  please  list 

on  a  separate  page  and  attach  to  the  critique  sheet. 


2,  Identify  any  key  issues  planned  for  analysis  that  are  not 
adequately  addressed  in  the  report.  Indicate  the  scope  of  the 
additional  analysis  needed. 


3.  How  can  the  methodology  used  to  conduct  the  study  be  Improved? 


4.  What  additional  Information  should  be  included  in  the  study  report 
to  more  clearly  demonstrate  the  bases  for  the  study  findings?  _ 


5.  How  can  the  study  findings  be  better  presented  to  support  the  needs 
of  both  action  officers  and  decisionmakers? 

- _ 


6.  How  can  the  written  material  in  the  report  be  Improved  in  terms  of 
clarity  of  presentation,  completeness,  and  style? 
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STUDY  CRITIQUE  (continued) 


7.  How  can  figures  and  tables  In  the  report  be  made  more  clear  and 
helpful? _ 


9.  How  win  the  findings  In  this  report  be  helpful  to  the  organization 
which  directed  that  the  work  be  done? 


If  they  win  not  be  helpful,  please  explain  why  not. 


10.  Judged  overall,  how  do  you  rate  the  study?  (circle  one) 
Poor  Fain  Average  Good 


Z 


8.  In  what  way  does  the  report  satisfy  the  expectations  that  were 


In  what  ways  does  the  report  fail  to  satisfy  the  expectations? 
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GLOSSARY 

ABBREVIATIONS,  ACRONYMS,  AND  SHORT  TERMS 


155  ER 

155mm  extended  range  round 

155  LtWt 

155mm  lightweight  howitzer 

155  TGP 

155mm  terminally  guided  projectile 

AAWS-M 

Advanced  Antitank  Weapons  System  -  Medium 

AC 

Active  Component 

ACP 

Army  cost  position 

ACR 

armored  cavalry  regiment 

ADATS 

Air  Defense  Antitank  System 

ADCSOPS 

Assistant  Deputy  Chief  of  Staff  for  Operations  and  Plans 

AFAS 

Advanced  Field  Artillery  System 

AFP 

Analysis  of  Force  Potential 

AGS 

Armored  Gun  System 

AHP 

analytical  hierarchy  process 

AMC 

US  Army  Materiel  Command 

AMS-H 

Advanced  Missile  System  -  Heavy 

ARIM 

Army  Resource  Integration  and  Management  (study) 

ARSTAF 

Army  Staff 

ARV 

armored  recovery  vehicle 

ASA(RI’A) 

Assistant  Secretary  of  the  Army  (Research,  Development,  and 
Acquisition 

ASF 

Army  stock  fund 

ASIOE 

associated  support  item(s)  of  equipment 

ASM 

armored  systems  modernization 

AT 

antitank 

ATACMS 

Army  Tactical  Missile  System 

Glossary- 1 


CAA-SR-92-10 


ATAS 

ATOM 

AVLB 

Adv  SEMA 

BAT 

BCE 

BPS 

BLK 

BOIP 

C2 

C4I 

CAA 

CEAC 

CPE 

CPV 

Cl 

CIS 

CITY 

CMV 

COPM 

CORBAN 

CR 

CSA 

CS 

CSS 

DAB 

DAS 


air-to-air  STINGER 

antitank  guided  missile 

armored  vehicle  launched  bridge 

advanced  special  electronic  mission  aircraft 

brilliant  antiarmor  technology 

baseline  cost  estimate 

Blue  force  surviving 

block 

basis  of  issue  plan 
command  and  control 

command,  control,  communications,  computers,  and  ‘.ntelligence 

US  Army  Concepts  Analysis  Agency 

US  Army  Cost  and  Economic  Analysis  Center 

Conventional  Porces  Europe 

cavalry  fighting  vehicle 

consistency  index 

Confederation  of  Independent  (former  Soviet)  States 

Commander’s  Independent  Thermal  Viewer 

combat  mobility  vehicle 

correlation  of  forces  and  means 

Corps  Battle  Analyzer  (model) 

consistency  ratio 

Chief  of  Staff  of  the  Army 

combat  support 

combat  service  support 

Defense  Acquisitinn  Board 

Director  of  the  Artiiy  Staff 
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DCSINT 

DCSLOG 

DCSOPS 

DCSPER 

DCSRM 

DMZ 

DOD 

DPAE 

DPICM 

DUSA(OR) 

EEA 

EPP 

FA 

FAADC21 

FAASV 

FARP 

FARV-A 

FAS 

FDT 

FER 

FIFV 

FMTV  2.5/5 

FUC 

FY 

GBCS-L/H 

GBS 


Deputy  Chief  of  Staff  for  Intelligence 

Deputy  Chief  of  Staff  for  Logistics 

Deputy  Chief  of  Staff  for  Operations  and  Plans 

Deputy  Chief  of  Staff  for  Personnel 

Deputy  Chief  of  Staff  for  Resource  Management 

demilitarized  zone 

Department  of  Defense 

Director,  Program  Analysis  and  Evaluation 

dual-purpose  improved  conventional  munition 

Deputy  Under  Secretary  of  the  Army  for  Operations  Research 

essential  element(s)  of  analysis 

Extended  Planning  Period 

field  artillery 

forward  area  air  defense  system  command,  control,  and 
intelligence 

field  artillery  ammunition  support  vehicle 

forward  arming  and  refueling  point 

future  armored  resupply  vehicle  -  artillery 

Force  Accounting  System 

first  destination  transportation 

force  exchange  ratio 

future  infantry  fighting  vehicle 

family  of  medium  tactical  vehicles  2.5-ton/5-ton 

first  unit  cost 

fiscal  year 

ground  based  common  sensor  -  lighL'heavy 
ground  based  sensor 
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GS  general  support 

GSR  general  support  role 

HAB  heavy  assault  bridge 

HET  heavy  equipment  transporter 

HMMWV  high  mobility,  multipurpose  wheeled  vehicle 

HQDA  Headquarters,  Department  of  the  Army 

IFV  infantry  fighting  vehicle 

IGB  inter-German  border 

IMG  information  management  office 

IMSL  international  mathematical  and  statistical  library 

ITV  Improved  TOW  vehicle 

JSTARS  Joint  Surveillance  Target  Attack  Radar  System 

LCCM  Life  Cycle  Cost  Model 

LER  loss  exchange  ratio 

LIN  line  item  number 

LOSAT  line  of  sight  antitank 

LP  linear  programming 

LRRDAP  Long-Range  Research,  Development,  and  Acquisition  Plan 

M  millions 

MACOM  major  Army  command 

MADM  multiple  attribute  decisionmaking 

MCA  Military  Construction,  Army 

MDEP  management  decision  package 

MIP  mixed  integer  program 

MISM  major  item  system  map 

MLRS  SADARM  multiple  launch  rocket  system  sense  and  dcbtruy  armor 
MLKS  1  GW  multiple  launch  rocket  system  terminally  guided  weapon 
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MiSS 

mission  needs  statement 

MOE 

measure(s)  of  effectiveness 

MPA 

Military  Procurement,  Army 

MPR 

maximum  production  rate 

MRC-E 

Major  Regional  Contingency  -  East 

MRC-W 

Major  Regional  Contingency  -  West 

MRL 

multiple  rocket  launcher 

MSR 

minimum  sustaining  rate 

NATO 

North  Atlantic  Treaty  Organization 

NBC 

nuclear,  biological,  and  chemical 

NEA 

Northeast  Asia 

NLOS 

nonline  of  sight 

NLOS-AT 

nonline  of  sight  -  antitank 

NLOS-CA 

nonline  of  sight  -  combined  arms 

O&S 

operating  and  support 

OBV 

obstacle  breach  vehicle 

ODCSOPS 

Office  of  the  Deputy  Chief  of  Staff  for  Operations  and  Plans 

OMA 

Operation  and  Maintenance,  Army 

OPA 

Other  Procurement,  Army 

OPTEMPO 

operational  tempo 

ORSA 

operations  research  systems  analysis 

OSD 

Office  of  the  Secretary  of  Defense 

OSE 

organization  support  equipment 

OSL 

Optimization  Software  Library 

PA 

Procurement,  Army 

PAED 

Program  Analysis  and  Evaluation  Directorate 

PC 

personal  computer 
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PE 

program  element 

PEG 

Program  Executive  Office 

PLS 

palletized  loading  system 

PM 

Program  Manager 

POM 

Program  Objective  Memorandum 

PPBES 

Planning,  Programming,  Budgeting,  and  Execution  System 

PROBE 

Program  Optimization  and  Budget  Evaluation 

Pk 

probability  of  kill 

QRA 

quick  reaction  analysis 

qty 

quantity 

RAM 

reliability,  availability,  and  maintainability 

RDA 

research,  development,  and  acquisition 

RDAISA 

Research,  Development,  and  Acquisition  Information  System 
Agency 

RDTE 

research,  development,  test,  and  evaluation 

RFM 

Red  force  movement 

ROC 

required  operational  capability 

ROF 

rate  of  fire 

RSM 

response  surface  methodology 

SAD ARM 

sense  and  destroy  armor 

SARDA 

Secretary  of  the  Army  for  Research,  Development,  and 
Acquisition 

SDT 

second  destination  transportation 

SELCOM 

Select  Committee 

SHORAD 

short-range  air  defense 

SIAM 

secondary  impact  analysis  modi  fier(s) 

SIMA 

Systems  Integration  and  Management  Activity 

SME 

subject  matter  expert 
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SSN 

standard  study  number 

SSPK 

single  shot  probability  of  kill 

SWA 

Southwest  Asia 

TAA 

Total  Army  Analysis  (study) 

TACJAM 

tactical  jamming 

TAEDP 

total  Army  equipment  distribution  program 

TAFCS 

The  Army  Force  Cost  System 

TDA 

table(s)  of  distribution  and  allowances 

TGP 

terminally  guided  projectile 

TGW 

terminally  guided  weapon 

TOA 

total  obligational  authority 

TOE 

table(s)  of  organization  and  equipment 

TOPSIS 

Technique  for  Order  Preference  by  Similarity  to  Ideal  Solution 
(model) 

TOW 

tube-launched,  optically  tracked,  wire-guided 

TOW2B 

tube-launched,  optically  tracked,  wire-guided  missile  2B 

TRADOC 

US  Army  Training  and  Doctrine  Command 

USMA 

US  Army  Military  Academy 

VAA 

Value  Added  Analysis 

VAAC 

Value  Added  Analysis  Capability 

VALOR 

Value  Added  Linear  Optimization  of  Resources  (model) 

VEMASID 

vehicle  magnetic  signal  duplicator 

VIC 

Vector-In-Commander  (model) 

WAM 

wide  area  mine 

WTCV 

wheeled  and  tracked  vehicles 
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THE  REASON  FOR  PERFORMING  THE  STUDY  was  to  provide  the  Director  for  Program 
Analysis  and  Evaluation,  and  the  Deputy  Chief  of  Staff  for  Operations  and  Plans 
(DCSOPS)  an  analytical  methodology  and  capability  to  support  the  development  of  a 
balanced  and  effective  Army  Program. 

THE  STUDY  SPONSORS  are  the  Director  for  Program  Analysis  and  Evaluation 
(DPAE),  Office  of  the  Chief  of  Staff,  Army,  and  the  Technical  Advisor,  Office  of  the 
Deputy  Chief  of  Staff  for  Operations  and  Plans  (ODCSOPS),  Headquarters, 
Department  of  the  Army  (HQDA). 

THE  STUDY  OBJECTIVES  were  to; 

(1)  Produce  VAA  coefficients  and  feasible  acquisition  alternatives  for  major  item 
systems  proposed  by  Headquarters,  Department  of  the  Army  (HQDA)  Long-Range 
Research,  Development,  and  Acquisition  Plan  (LRRDAP)  and  constrained  by 
modernization  total  obligational  authority  (TO A).  The  process  must  measure  and 
analyze  the  capability  of  US  Army  forces  to  conduct  conventional  operations  in 
scenarios  consistent  with  the  Illustrative  Planning  Scenarios  of  the  Defense  Planning 
Guidance,  Fiscal  Year  (FY)  1994-1999. 

(2)  Identify  and  develop  a  Value  Added  Analysis  Capability  (VAAC)  to  include  all 
appropriate  hardware,  software,  and  interfaces.  The  VAAC  must  tap  major 
authoritative  Army  data  bases  such  as  the  Total  Army  Equipment  Distribution 
Program  (TAEDP),  Force  Accounting  System  (FAS),  and  the  Army  Force  Cost 
System  (TAFCS). 

(3)  Identify  or  develop  models  and  techniques  that  support  the  VAA  methodology. 
The  VAAC  and  related  models  must  be  capable  of  operating  in  a  "quick  turnaround" 
environment,  defined  as  1  week  or  less. 

(4)  Conduct  a  demonstration  of  the  refined  methodology  and  VAAC  prior  to  the 
building  of  the  1994-1999  Program  Objective  Memorandum  (POM). 

(5)  Continue  the  refinement  and  implementation  of  the  VAA  methodology  for 
estimating  the  value  of  either  competing  major  item  systems  or  management 
decision  packages  (MDEPs)  to  the  Total  Army  Program. 

THE  SCOPE  OF  THE  STUDY  included  the  research,  development,  and  acquisition 
(RDA)  appropriation  for  selected  major  item  systems  in  FY  1994  and  FY  2008. 

THE  MAIN  ASSUMPTION  of  this  study  is  that  HQDA  needs  a  relatively  quick  method 
for  conducting  program  tradeoffs  which  has  sound  analytical  underpinnings. 


THE  BASIC  APPROACH  of  this  study  was  to: 


(1)  Enhance  and  expand  the  analytic  approach  for  program  issue  tradeoffs 
developed  in  Phase  I  of  the  VAA  study  effort. 

(2)  Develop  a  VAAC  for  implementing  the  methodology  to  include  software 
modules  where  appropriate. 

(3)  Demonstrate  the  VAAC  using  issues  from  the  94-99  POM  issue  cycle. 

THE  PRINCIPAL  FINDINGS  of  the  study  were: 

(1)  The  Value  Added  Analysis  study  framework,  as  developed  in  the  VAA  Phase  I 
Study,  was  shown  to  be  useful  in  evaluating  POM  issues. 

(2)  The  use  of  an  experimental  design  and  a  response  surface  methodology  was 
found  to  be  an  effective  means  to  determine  system  contribution  to  combat  results. 

(3)  The  Life  Cycle  Cost  Model  (LCCM)  is  a  useful  tool  for  providing  action  officers 
with  detailed  cost  estimates  for  candidate  POM  programs. 

(4)  The  development  of  a  mixed  integer  programming  formulation  that  allowed 
consideration  of  cost/quantity  relationships  and  handles  fixed  production  costs  and 
research,  development,  test,  and  evaluation  costs  explicitly  was  shown  to  be  an 
extremely  effective  method  of  cost-benefit  analysis. 

(5)  Additional  research  is  required  to  find  an  improved  method  of  effectiveness 
integration  to  replace  the  Technique  for  Order  Preference  by  Similarity  to  Ideal 
Solution  (TOPSIS)  Model. 

THE  STUDY  EFFORT  was  directed  by  LTC  Robert  R.  Koury,  Force  Systems 
Directorate,  US  Army  Concepts  Analysis  Agency  (CAA). 

COMMENTS  AND  QUESTIONS  may  be  sent  to  the  Director,  US  Army  Concepts 
Analysis  Agency,  ATTN:  CSCA-FSR,  8120  Woodmont  Avenue,  Bethesda,  Maryland 
20814-2797. 
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THE  REASON  FOR  PERFORMING  THE  STUDY  was  to  provide  the  Director  for  Program 
Analysis  and  Evaluation,  and  the  Deputy  Chief  of  Staff  for  Operations  and  Plans 
(DCSOPS)  an  analytical  methodology  and  capability  to  support  the  development  of  a 
balanced  and  effective  Army  Program. 

THE  STUDY  SPONSORS  are  the  Director  for  Program  Analysis  and  Evaluation 
(DPAE),  Office  of  the  Chief  of  Staff,  Army,  and  the  Technical  Advisor,  Office  of  the 
Deputy  Chief  of  Staff  for  Operations  and  Plans  (ODCSOPS),  Headquarters, 
Department  of  the  Army  (HQDA). 

THE  STUDY  OBJECTIVES  were  to; 

(1)  Produce  VAA  coefficients  and  feasible  acquisition  alternatives  for  major  item 
systems  proposed  by  Headquarters,  Department  of  the  Army  (HQDA)  Long-Range 
Research,  Development,  and  Acquisition  Plan  (LRRDAP)  and  constrained  by 
modernization  total  obligational  authority  (TOA).  The  process  must  measure  and 
analyze  the  capability  of  US  Army  forces  to  conduct  conventional  operations  in 
scenarios  consistent  with  the  Illustrative  Planning  Scenarios  of  the  Defense  Planning 
Guidance,  Fiscal  Year  (FY)  1994-1999. 

(2)  Ider  Ufy  and  develop  a  Value  Added  Analysis  Capability  (VAAC)  to  include  all 
appropriate  hardware,  software,  and  interfaces.  The  VAAC  must  tap  major 
authoritative  Army  data  bases  such  as  the  Total  Army  Equipment  Distribution 
Program  (TAEDP),  Force  Accounting  System  (FAS),  and  the  Army  Force  Cost 
System  (TAFCS). 

(3)  Ider  tify  or  develop  models  and  techniques  that  support  the  VAA  methodology. 
The  VAAC  and  related  models  must  be  capable  of  operating  in  a  "quick  turnaround" 
environment,  defined  as  1  week  or  less. 

(4)  Con  -.uct  a  demonstration  of  the  refined  methodology  and  VAAC  prior  to  the 
building  of  ^he  1994-1999  Program  Objective  Memorandum  (POM). 

(5)  Continue  the  refinement  and  implementation  of  the  VAA  methodology  for 
estimating  the  value  of  either  competing  major  item  systems  or  management 
decision  packages  (MDEPs)  to  the  Total  Army  Program. 

THE  SCOPE  OF  THE  STUDY  included  the  research,  development,  and  acquisition 
(RDA)  appropriation  for  selected  major  item  systems  in  FY  1994  and  FY  2008. 

THE  MAIN  ASSUMPTION  of  this  study  is  that  HQDA  needs  a  relatively  quick  method 
for  conducting  program  tradeoffs  which  has  sound  analytical  underpinnings. 


THE  BASIC  APPROACH  of  this  study  was  to: 

(1)  Enhance  and  expand  the  analytic  approach  for  program  issue  tradeoffs 
developed  in  Phase  I  of  the  VAA  study  effort. 

(2)  Develop  a  VAAC  for  implementing  the  methodology  to  include  software 
modules  where  appropriate. 

(3)  Demonstrate  the  VAAC  using  issues  from  the  94-99  POM  issue  cycle. 

THE  PRINCIPAL  FINDINGS  of  the  study  were: 

(1)  The  Value  Added  Analysis  study  framework,  as  developed  in  the  VAA  Phase  I 
Study,  was  shown  to  be  useful  in  evaluating  POM  issues. 

(2)  The  use  of  an  experimental  design  and  a  response  surface  methodology  was 
found  to  be  an  effective  means  to  determine  system  contribution  to  combat  results. 

(3)  The  Life  Cycle  Cost  Model  (LCCM)  is  a  useful  tool  for  providing  action  officers 
with  detailed  cost  estimates  for  candidate  POM  programs. 

(4)  The  development  of  a  mixed  integer  programming  formulation  that  allowed 
consideration  of  cost/quantity  relationships  and  handles  fixed  production  costs  and 
research,  development,  test,  and  evaluation  costs  explicitly  was  shown  to  be  an 
extremely  effective  method  of  cost-benefit  analysis. 

(5)  Additional  research  is  required  to  find  an  improved  method  of  effectiveness 
integration  to  replace  the  Technique  for  Order  Preference  by  Similarity  to  Ideal 
Solution  (TOPSIS)  Model. 

THE  STUDY  EFFORT  was  directed  by  LTC  Robert  R,  Koury,  Force  Systems 
Directorate,  US  Army  Concepts  Analysis  Agency  (CAA). 

COMMENTS  AND  QUESTIONS  may  be  sent  to  the  Director,  US  Army  Concepts 
Analysis  Agency,  ATTN:  CSCA-FSR,  8120  Woodmont  Avenue,  Bethesda,  Maryland 
20814-2797. 
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THE  REASON  FOR  PERFORMING  THE  STUDY  was  to  provide  the  Director  for  Program 
Analysis  and  Evaluation,  and  the  Deputy  Chief  of  Staff  for  Operations  and  Plans 
(DCSOPS)  an  analytical  methodology  and  capability  to  support  the  development  of  a 
balanced  and  effective  Army  Program. 

THE  STUDY  SPONSORS  are  the  Director  for  Program  Analysis  and  Evaluation 
(DPAE),  Office  of  the  Chief  of  Staff,  Army,  and  the  Technical  Advisor,  Office  of  the 
Deputy  Chief  of  Staff  for  Operations  and  Plans  (ODCSOPS),  Headquarters, 
Department  of  the  Army  (HQDA). 

THE  STUDY  OBJECTIVES  were  to: 

(1)  Produce  VAA  coefficients  and  feasible  acquisition  alternatives  for  major  item 
systems  proposed  by  Headquarters,  Department  of  the  Army  (HQDA)  Long-Range 
Research,  Development,  and  Acquisition  Plan  (LRRDAP)  and  constrained  by 
modernization  total  obligational  authority  (TO  A).  The  process  must  measure  and 
analyze  the  capability  of  US  Army  forces  to  conduct  conventional  operations  in 
scenarios  consistent  with  the  Illustrative  Planning  Scenarios  of  the  Defense  Planning 
Guidance,  Fiscal  Year  (FY)  1994-1999. 

(2)  Identify  and  develop  a  Value  Added  Analysis  Capability  (VAAC)  to  include  all 
appropriate  hardware,  software,  and  interfaces.  The  VAAC  must  tap  major 
authoritative  Army  data  bases  such  as  the  Total  Army  Equipment  Distribution 
Program  (TAEDP),  Force  Accounting  System  (FAS),  and  the  Army  Force  Cost 
System  (TAFCS). 

(3)  Identify  or  develop  models  and  techniques  that  support  the  VAA  methodology. 
The  VAAC  and  related  models  must  be  capable  of  operating  in  a  "quick  turnaround" 
environment,  defined  as  1  week  or  less. 

(4)  Conduct  a  demonstration  of  the  refined  methodology  and  VAAC  prior  to  the 
building  of  the  1994-1999  Program  Objective  Memorandum  (POM). 

(5)  Continue  the  refinement  and  implementation  of  the  VAA  methodology  for 
estimating  the  value  of  either  competing  major  item  systems  or  management 
decision  packages  (MDEPs)  to  the  Total  Army  Program. 

THE  SCOPE  OF  THE  STUDY  included  the  research,  development,  and  acquisition 
(RDA)  appropriation  for  selected  major  item  systems  in  FY  1994  and  FY  2008. 

THE  MAIN  ASSUMPTION  of  this  study  is  that  HQDA  needs  a  relatively  quick  method 
for  conducting  program  tradeoffs  which  has  sound  analytical  underpinnings. 


THE  BASIC  APPROACH  of  this  study  was  to: 


(1)  Enhance  and  expand  the  analytic  approach  for  program  issue  tradeoffs 
developed  in  Phase  I  of  the  VAA  study  effort. 

(2)  Develop  a  VAAC  for  implementing  the  methodology  to  include  software 
modules  where  appropriate. 

(3)  Demonstrate  the  VAAC  using  issues  from  the  94-99  POM  issue  cycle. 

THE  PRINCIPAL  FINDINGS  of  the  study  were: 

(1)  The  Value  Added  Analysis  study  framework,  as  developed  in  the  VAA  Phase  I 
Study,  was  shown  to  be  useful  in  evaluating  POM  issues. 

(2)  The  use  of  an  experimental  design  and  a  response  surface  methodology  was 
found  to  be  an  effective  means  to  determine  system  contribution  to  combat  results. 

(3)  The  Life  Cycle  Cost  Model  (LCCM)  is  a  useful  tool  for  providing  action  officers 
with  detailed  cost  estimates  for  candidate  POM  programs. 

(4)  The  development  of  a  mixed  integer  programming  formulation  that  allowed 
consideration  of  cost/quantity  relationships  and  handles  fixed  production  costs  and 
research,  development,  test,  and  evaluation  costs  explicitly  was  shown  to  be  an 
extremely  effective  method  of  cost-benefit  analysis. 

(5)  Additional  research  is  required  to  find  an  improved  method  of  effectiveness 
integration  to  replace  the  Technique  for  Order  Preference  by  Similarity  to  Ideal 
Solution  (TOPSIS)  Model. 

THE  STUDY  EFFORT  was  directed  by  LTC  Robert  R.  Koury,  Force  Systems 
Directorate,  US  Army  Concepts  Analysis  Agency  (CAA). 

COMMENTS  AND  QUESTIONS  may  be  sent  to  the  Director,  US  Army  Concepts 
Analysis  Agency,  ATTN:  CSCA-FSR,  8120  Woodmont  Avenue,  Bethesda,  Maryland 
20814-2797. 


